| 490 | 0 | 119 |
| Downloads | Citas | Reads |
In light of the disparity between graphic and text modes and insufficient attention to textual information, a multimodal Chinese sarcasm detection model integrated with linguistic features was proposed. The Chi-square statistical method was used to extract words with sarcastic and non-sarcastic meanings, forming the linguistic feature system. TextCNN was utilized to extract linguistic features, enhancing the distinction between sarcastic and non-sarcastic characteristics. TextCNN and ResNet were employed to extract text and image features, and a cross-attention mechanism was introduced. Residual connections were used to fuse text and image features, to help preserve language characteristics. The effectiveness of the proposed model was verified by using an emergency multimodal dataset containing sarcastic comments. The results showed that the model outperformed the baseline model, and focusing on textual linguistic features helped improve the efficiency of problem-solving.
[1] ZHANG M,ZHANG Y,FU G.Tweet sarcasm detection using deep neural network[C]//Proceedings of the 26th International Conference on Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2016:2449-2460.
[2] BILAL G,JIHEN K,FARAH B,et al.Overview of the track on irony detection in Arabic tweets[C]//Proceedings of the 11th Annual Meeting of the Forum for Information Retrieval Evaluation.New York:ACM Press,2019:10-13.
[3] 张继东，蒋丽萍.基于多模态深度学习的旅游评论反讽识别研究[J].情报理论与实践，2022,45(7):158-164.ZHANG J D,JIANG L P.Research on irony recognition of travel reviews based on multi-modal deep learning[J].Information studies:theory & application,2022,45(7):158-164.
[4] 张亚洲，俞洋，朱少林，等.一种量子概率启发的对话讽刺识别网络模型[J].电子学报，2022,50(8):1885-1893.ZHANG Y Z,YU Y,ZHU S L,et al.A quantum probability inspired network for dialogue sarcasm recognition[J].Acta electronica sinica,2022,50(8):1885-1893.
[5] SCHIFANELLA R,JUAN P D,TETREAULT J,et al.Detecting sarcasm in multimodal social platforms[EB/OL].(2016-08-08)[2024-03-15].https://doi.org/10.48550/arXiv.1608.02289.
[6] WANG X Y,SUN X W,YANG T,et al.Building a bridge:a method for image-text sarcasm detection without pretraining on image-text data[C]//Proceedings of the 1st International Workshop on Natural Language Processing Beyond Text.Stroudsburg:Association for Computational Linguistics,2020:19-29.
[7] PAN H L,LIN Z,FU P,et al.Modeling intra and inter-modality incongruity for multi-modal sarcasm detection[C]//Findings of the Association for Computational Linguistics:EMNLP 2020.Stroudsburg:Association for Computational Linguistics,2020:1383-1392.
[8] GUPTA S,SHAH A,SHAH M,et al.FiLMing multimodal sarcasm detection with attention[C]//Proceedings of the International Conference on Neural Information Processing.Cham:Springer International Publishing,2021:178-186.
[9] LIANG B,LOU C W,LI X,et al.Multi-modal sarcasm detection with interactive in-modal and cross-modal graphs[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM Press,2021:4707-4715.
[10] WEN C S,JIA G L,YANG J F.DIP:dual incongruity perceiving network for sarcasm detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2023:2540-2550.
[11] TIAN Y,XU N,ZHANG R K,et al.Dynamic routing transformer network for multimodal sarcasm detection[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2023:2468-2480.
[12] CAI Y T,CAI H Y,WAN X J.Multi-modal sarcasm detection in Twitter with hierarchical fusion model[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2019:2506-2515.
[13] SANGWAN S,AKHTAR M S,BEHERA P,et al.I didn′t mean what I wrote! Exploring multimodality for sarcasm detection[C]//International Joint Conference on Neural Networks.Piscataway:IEEE Press,2020:1-8.
[14] XU N,ZENG Z X,MAO W J.Reasoning with multimodal sarcastic tweets via modeling cross-modality contrast and semantic association[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2020:3777-3786.
[15] LIANG B,LOU C W,LI X,et al.Multi-modal sarcasm detection via cross-modal graph convolutional network[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2022:1767-1777.
[16] 卢欣，李旸，王素格.融合语言特征的卷积神经网络的反讽识别方法[J].中文信息学报，2019,33(5):31-38.LU X,LI Y,WANG S G.Linguistic features enhanced convolutional neural networks for irony recognition[J].Journal of Chinese information processing,2019,33(5):31-38.
[17] TANG Y J,CHEN H.Chinese irony corpus construction and ironic corpus construction and ironic structure analysis[C]//Proceedings of the 25th International Conference on Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2014:1269-1278.
[18] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2014:1746-1751.
[19] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:770-778.
[20] 胡文彬，陈龙，黄贤波，等.融合交叉注意力的突发事件多模态中文反讽识别模型[J].智能系统学报，2024,19(2):392-400.HU W B,CHEN L,HUANG X B,et al.A multimodal Chinese sarcasm detection model for emergencies based on cross attention[J].CAAI transactions on intelligent systems,2024,19(2):392-400.
[21] WU S Y,SU E T,LEI B Y,et al.TextCNN-based text classification for E-government[C]//Proceedings of the 6th International Conference on Information Science and Control Engineering.Piscataway:IEEE Press,2019:929-934.
[22] SHARFUDDIN A A,TIHAMI M N,ISLAM M S.A deep recurrent neural network with BiLSTM model for sentiment classification[C]//International Conference on Bangla Speech and Language Processing.Piscataway:IEEE Press,2018:1-4.
[23] JACOB D,CHANG M W,KENTON L,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:Association for Computational Linguistics,2019:4171-4186.
[24] 陈俊松，易积政，陈爱斌.CNS-Net:一种循环多注意力特征聚合架构[J].郑州大学学报(理学版),2024,56(2):73-79.CHEN J S,YI J Z,CHEN A B.CNS-Net:a cyclic multi-attention feature aggregation architecture[J].Journal of Zhengzhou university (natural science edition),2024,56(2):73-79.
[25] 计咏梅.基于深度学习的卷积码盲识别方法研究[D].西安：西安电子科技大学，2023.JI Y M.Research on blind recognition method of convolutional code based on deep learning[D].Xi′an:Xidian University,2023.
Basic Information:
DOI：10.13705/j.issn.1671-6841.2024096
China Classification Code:TP391.1;TP183
Citation Information:
[1]HU Wenbin,CHEN Long,HAN Tianle ,et al.A Multimodal Chinese Sarcasm Detection Model Integrated with Linguistic Features[J].Journal of Zhengzhou University(Natural Science Edition),2025,57(05):16-23.DOI:10.13705/j.issn.1671-6841.2024096.
Fund Information:
国家自然科学基金项目(72174079); 江苏省“青蓝工程”优秀教学团队项目(2022-29)
2024-10-30
2024-10-30
2024-10-30