Exploring Multimodal Features for Sentiment Classification of Social Media Data

Sumana Biswas; Karen Young; Josephine Griffith

doi:10.1007/978-981-99-8324-7_44

Exploring Multimodal Features for Sentiment Classification of Social Media Data

Computer Science

University of Galway

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

1 Citation (Scopus)

Abstract

Effectively capturing and interpreting sentiments from image and text data is a challenge for the sentiment analysis task. While the expressive objects in an image that evokes human emotion are commonly explored in sentiment analysis, the object’s attributes often remain unexplored. In this paper, we describe the extraction of objects, features, and attributes from images and text individually and in different combinations. We conducted extensive experiments, using two different datasets, to evaluate the performance of sentiment analysis models across these features. We demonstrate the best multimodal features across image attributes and text features that can be used to classify the sentiment. We also identify the efficacy of the CNN and Fusion models among the four considered models. This study contributes to sentiment analysis by utilizing untapped attributes of objects in images and demonstrating the advantages of combining features from multimodal data using deep learning models.

Original language	English
Title of host publication	Proceedings of International Conference on Information Technology and Applications - ICITA 2023
Editors	Abrar Ullah, Sajid Anwar, Davide Calandra, Raffaele Di Fuccio
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	527-537
Number of pages	11
ISBN (Print)	9789819983230
DOIs	https://doi.org/10.1007/978-981-99-8324-7_44
Publication status	Published - 2024
Event	17th International Conference on Information Technology and Applications, ICITA 2023 - Lisbon, Portugal Duration: 20 Oct 2022 → 22 Oct 2022

Publication series

Name	Lecture Notes in Networks and Systems
Volume	839
ISSN (Print)	2367-3370
ISSN (Electronic)	2367-3389

Conference

Conference	17th International Conference on Information Technology and Applications, ICITA 2023
Country/Territory	Portugal
City	Lisbon
Period	20/10/22 → 22/10/22

Keywords

Deep learning
Multimodal features
Objects’ attributes
Sentiment analysis
Social media

Access to Document

10.1007/978-981-99-8324-7_44

Cite this

Biswas, S., Young, K., & Griffith, J. (2024). Exploring Multimodal Features for Sentiment Classification of Social Media Data. In A. Ullah, S. Anwar, D. Calandra, & R. Di Fuccio (Eds.), Proceedings of International Conference on Information Technology and Applications - ICITA 2023 (pp. 527-537). (Lecture Notes in Networks and Systems; Vol. 839). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-8324-7_44

Biswas, Sumana ; Young, Karen ; Griffith, Josephine. / Exploring Multimodal Features for Sentiment Classification of Social Media Data. Proceedings of International Conference on Information Technology and Applications - ICITA 2023. editor / Abrar Ullah ; Sajid Anwar ; Davide Calandra ; Raffaele Di Fuccio. Springer Science and Business Media Deutschland GmbH, 2024. pp. 527-537 (Lecture Notes in Networks and Systems).

@inproceedings{62e74eab84bc45d387e576a979b7087d,

title = "Exploring Multimodal Features for Sentiment Classification of Social Media Data",

abstract = "Effectively capturing and interpreting sentiments from image and text data is a challenge for the sentiment analysis task. While the expressive objects in an image that evokes human emotion are commonly explored in sentiment analysis, the object{\textquoteright}s attributes often remain unexplored. In this paper, we describe the extraction of objects, features, and attributes from images and text individually and in different combinations. We conducted extensive experiments, using two different datasets, to evaluate the performance of sentiment analysis models across these features. We demonstrate the best multimodal features across image attributes and text features that can be used to classify the sentiment. We also identify the efficacy of the CNN and Fusion models among the four considered models. This study contributes to sentiment analysis by utilizing untapped attributes of objects in images and demonstrating the advantages of combining features from multimodal data using deep learning models.",

keywords = "Deep learning, Multimodal features, Objects{\textquoteright} attributes, Sentiment analysis, Social media",

author = "Sumana Biswas and Karen Young and Josephine Griffith",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.; 17th International Conference on Information Technology and Applications, ICITA 2023 ; Conference date: 20-10-2022 Through 22-10-2022",

year = "2024",

doi = "10.1007/978-981-99-8324-7\_44",

language = "English",

isbn = "9789819983230",

series = "Lecture Notes in Networks and Systems",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "527--537",

editor = "Abrar Ullah and Sajid Anwar and Davide Calandra and \{Di Fuccio\}, Raffaele",

booktitle = "Proceedings of International Conference on Information Technology and Applications - ICITA 2023",

}

Biswas, S, Young, K & Griffith, J 2024, Exploring Multimodal Features for Sentiment Classification of Social Media Data. in A Ullah, S Anwar, D Calandra & R Di Fuccio (eds), Proceedings of International Conference on Information Technology and Applications - ICITA 2023. Lecture Notes in Networks and Systems, vol. 839, Springer Science and Business Media Deutschland GmbH, pp. 527-537, 17th International Conference on Information Technology and Applications, ICITA 2023, Lisbon, Portugal, 20/10/22. https://doi.org/10.1007/978-981-99-8324-7_44

Exploring Multimodal Features for Sentiment Classification of Social Media Data. / Biswas, Sumana; Young, Karen ; Griffith, Josephine.
Proceedings of International Conference on Information Technology and Applications - ICITA 2023. ed. / Abrar Ullah; Sajid Anwar; Davide Calandra; Raffaele Di Fuccio. Springer Science and Business Media Deutschland GmbH, 2024. p. 527-537 (Lecture Notes in Networks and Systems; Vol. 839).

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

TY - GEN

T1 - Exploring Multimodal Features for Sentiment Classification of Social Media Data

AU - Biswas, Sumana

AU - Young, Karen

AU - Griffith, Josephine

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

PY - 2024

Y1 - 2024

N2 - Effectively capturing and interpreting sentiments from image and text data is a challenge for the sentiment analysis task. While the expressive objects in an image that evokes human emotion are commonly explored in sentiment analysis, the object’s attributes often remain unexplored. In this paper, we describe the extraction of objects, features, and attributes from images and text individually and in different combinations. We conducted extensive experiments, using two different datasets, to evaluate the performance of sentiment analysis models across these features. We demonstrate the best multimodal features across image attributes and text features that can be used to classify the sentiment. We also identify the efficacy of the CNN and Fusion models among the four considered models. This study contributes to sentiment analysis by utilizing untapped attributes of objects in images and demonstrating the advantages of combining features from multimodal data using deep learning models.

AB - Effectively capturing and interpreting sentiments from image and text data is a challenge for the sentiment analysis task. While the expressive objects in an image that evokes human emotion are commonly explored in sentiment analysis, the object’s attributes often remain unexplored. In this paper, we describe the extraction of objects, features, and attributes from images and text individually and in different combinations. We conducted extensive experiments, using two different datasets, to evaluate the performance of sentiment analysis models across these features. We demonstrate the best multimodal features across image attributes and text features that can be used to classify the sentiment. We also identify the efficacy of the CNN and Fusion models among the four considered models. This study contributes to sentiment analysis by utilizing untapped attributes of objects in images and demonstrating the advantages of combining features from multimodal data using deep learning models.

KW - Deep learning

KW - Multimodal features

KW - Objects’ attributes

KW - Sentiment analysis

KW - Social media

UR - https://www.scopus.com/pages/publications/85189543438

U2 - 10.1007/978-981-99-8324-7_44

DO - 10.1007/978-981-99-8324-7_44

M3 - Conference Publication

AN - SCOPUS:85189543438

SN - 9789819983230

T3 - Lecture Notes in Networks and Systems

SP - 527

EP - 537

BT - Proceedings of International Conference on Information Technology and Applications - ICITA 2023

A2 - Ullah, Abrar

A2 - Anwar, Sajid

A2 - Calandra, Davide

A2 - Di Fuccio, Raffaele

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th International Conference on Information Technology and Applications, ICITA 2023

Y2 - 20 October 2022 through 22 October 2022

ER -

Biswas S, Young K , Griffith J. Exploring Multimodal Features for Sentiment Classification of Social Media Data. In Ullah A, Anwar S, Calandra D, Di Fuccio R, editors, Proceedings of International Conference on Information Technology and Applications - ICITA 2023. Springer Science and Business Media Deutschland GmbH. 2024. p. 527-537. (Lecture Notes in Networks and Systems). doi: 10.1007/978-981-99-8324-7_44

Exploring Multimodal Features for Sentiment Classification of Social Media Data

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this