Sarcasm is a linguistic phenomenon indicating a discrepancy between literal meanings and implied intentions. Due to its sophisticated nature, it is usually challenging to be detected from the text itself. As a result, multi-modal sarcasm detection has received more attention in both academia and industries. However, most existing techniques only modeled the atomic-level inconsistencies between the text input and its accompanying image, ignoring more complex compositions for both modalities. Moreover, they neglected the rich information contained in external knowledge, e.g., image captions. In this paper, we propose a novel hierarchical framework for sarcasm detection by exploring both the atomic-level congruity based on multi-head cross attention mechanism and the composition-level congruity based on graph neural networks, where a post with low congruity can be identified as sarcasm. In addition, we exploit the effect of various knowledge resources for sarcasm detection. Evaluation results on a public multi-modal sarcasm detection dataset based on Twitter demonstrate the superiority of our proposed model.
MUStReason: A Benchmark for Diagnosing Pragmatic Reasoning in Video-LMs for Multimodal Sarcasm Detection
Max Planck Institute for Informatics, Saarland Informatics Campus（马克斯·普朗克研究所信息学研究所，萨尔兰州信息学校园）；Saarland University（萨尔兰州大学）；The University of Edinburgh（爱丁堡大学）；Samsung AI Center, Cambridge（三星人工智能中心，剑桥）
MMSD2.0: Towards a Reliable Multi-modal Sarcasm Detection System
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Laboratory（鹏城实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）
A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis
Beijing Institute of Technology（北京理工大学）；The Open University（开放大学）；The Hong Kong Polytechnic University（香港理工大学）；Chongqing University（重庆大学）；China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司）；Nanjing University（南京大学）；Zhengzhou University of Light Industry（郑州轻工业大学）
IRONIC: Coherence-Aware Reasoning Chains for Multi-Modal Sarcasm Detection
College of Information Sciences and Technology（信息科学与技术学院）；The Pennsylvania State University（宾夕法尼亚州立大学）；Department of Computer Science and Engineering（计算机科学与工程系）；National Institute of Technology, Tiruchirappalli（特里奇里帕利理工学院）
GDCNet: Generative Discrepancy Comparison Network for Multimodal Sarcasm Detection
State Key Laboratory of AI Safety, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS)（人工智能安全国家重点实验室，计算技术研究所（ICT），中国科学院（CAS））；University of Chinese Academy of Sciences, CAS（中国科学院大学）；Pengcheng Laboratory（鹏城实验室）；Shenzhen Stock Exchange（深圳证券交易所）
SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning
Speech Technology Lab, University of Groningen, The Netherlands（格罗宁根大学语音技术实验室，荷兰）；Center for Language and Cognition, University of Groningen, The Netherlands（格罗宁根大学语言与认知中心，荷兰）
Multi-View Incongruity Learning for Multimodal Sarcasm Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；State Key Laboratory of Software Development Environment, Beihang University（北京航空航天大学软件开发环境国家重点实验室）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机学院）；Department of Computer Science, University of Illinois at Chicago（伊利诺伊大学芝加哥分校计算机系）
StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件学院）；Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；City University of Macau（澳门城市大学）
SemIRNet: A Semantic Irony Recognition Network for Multimodal Sarcasm Detection
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Southern California（南加州大学）；Kazan Federal University（卡扎那夫联邦大学）
Multi-modal Semantic Understanding with Contrastive Cross-modal Feature Alignment
Peking University（北京大学）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；School of Computer Science, Peking University（北京大学计算机学院）
Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding
Peking University（北京大学）；National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；School of Computer Science, Peking University（北京大学计算机学院）
