A Multimodal Sentiment Classification Study Based on Mvsa-Single Dataset

College of Mathematics and Informatics & College of Software Engineering, South China Agricultural University, Guangzhou, Guangdong, China

kyx@stu.scau.edu.cn

Abstract

Multimodal sentiment classification refers to the task of analyzing and identifying textual, visual, and other multimodal data using deep learning techniques. This study conducts an in-depth exploration of the single-stream multimodal emotion classification model based on the intermodal attention mechanism, the dual-stream multimodal emotion classification model, the MCAN emotion classification model, and the original emotion classification model without the addition of the intermodal attention mechanism. The architecture, training method and data preprocessing strategy of each model are elaborated. This paper compares and analyze the performance of each model on the Multi-View Sentiment Analysis (MVSA) dataset in terms of accuracy, precision, recall, AUC and PR curve. The results show that the different models have their own advantages and disadvantages, the single-stream model has an advantage in recall but lower precision, the dual-stream model shows a unique ability in precision but poorer performance in recall, and the MCAN model has a better overall metrics, and all the three models have a higher accuracy than the original model. This study provides a valuable reference for the selection and optimization of models in the field of multimodal sentiment analysis, and also points out the direction for further exploring the application of multimodal fusion technology.