Redirigiendo al acceso original de articulo en 17 segundos...
Inicio  /  Applied Sciences  /  Vol: 12 Par: 21 (2022)  /  Artículo
ARTÍCULO
TITULO

EFAFN: An Efficient Feature Adaptive Fusion Network with Facial Feature for Multimodal Sarcasm Detection

Yukuan Sun    
Hangming Zhang    
Shengjiao Yang and Jianming Wang    

Resumen

Sarcasm often manifests itself in some implicit language and exaggerated expressions. For instance, an elongated word, a sarcastic phrase, or a change of tone. Most research on sarcasm detection has recently been based on text and image information. In this paper, we argue that most image data input to the sarcasm detection model is redundant, for example, complex background information and foreground information irrelevant to sarcasm detection. Since facial details contain emotional changes and social characteristics, we should pay more attention to the image data of the face area. We, therefore, treat text, audio, and face images as three modalities and propose a multimodal deep-learning model to tackle this problem. Our model extracts the text, audio, and image features of face regions and then uses our proposed feature fusion strategy to fuse these three modal features into one feature vector for classification. To enhance the model?s generalization ability, we use the IMGAUG image enhancement tool to augment the public sarcasm detection dataset MUStARD. Experiments show that although using a simple supervised method is effective, using a feature fusion strategy and image features from face regions can further improve the F1 score from 72.5% to 79.0%.

 Artículos similares

       
 
Qiyan Li, Zhi Weng, Zhiqiang Zheng and Lixin Wang    
The decrease in lake area has garnered significant attention within the global ecological community, prompting extensive research in remote sensing and computer vision to accurately segment lake areas from satellite images. However, existing image segmen... ver más
Revista: Applied Sciences

 
Qingyong Zhang, Changhuan Song and Yiqing Yuan    
Vehicle gearboxes are subject to strong noise interference during operation, and the noise in the signal affects the accuracy of fault identification. Signal denoising and fault diagnosis processes are often conducted independently, overlooking their syn... ver más
Revista: Applied Sciences

 
Pengfei Ning, Dianjun Zhang, Xuefeng Zhang, Jianhui Zhang, Yulong Liu, Xiaoyi Jiang and Yansheng Zhang    
The Array for Real-time Geostrophic Oceanography (Argo) program provides valuable data for maritime research and rescue operations. This paper is based on Argo historical and satellite observations, and inverted sea surface and submarine drift trajectori... ver más

 
Zhikai Jiang, Li Su and Yuxin Sun    
Accurate ship object detection ensures navigation safety and effective maritime traffic management. Existing ship target detection models often have the problem of missed detection in complex marine environments, and it is hard to achieve high accuracy a... ver más

 
Baobao Liu, Heying Wang, Zifan Cao, Yu Wang, Lu Tao, Jingjing Yang and Kaibing Zhang    
Defect detection holds significant importance in improving the overall quality of fabric manufacturing. To improve the effectiveness and accuracy of fabric defect detection, we propose the PRC-Light YOLO model for fabric defect detection and establish a ... ver más
Revista: Applied Sciences