In the domain of multimodal intent recognition (MIR), the objective is to recognize human intent by integrating a variety of modalities, such as language text, body gestures, and tones. However, existing approaches face difficulties adequately capturing the intrinsic connections between the modalities and overlooking the corresponding semantic representations of intent. To address these limitations, we present the Anchor-based Multimodal Embedding with Semantic Synchronization (A-MESS) framework. We first design an Anchor-based Multimodal Embedding (A-ME) module that employs an anchor-based embedding fusion mechanism to integrate multimodal inputs. Furthermore, we develop a Semantic Synchronization (SS) strategy with the Triplet Contrastive Learning pipeline, which optimizes the process by synchronizing multimodal representation with label descriptions produced by the large language model. Comprehensive experiments indicate that our A-MESS achieves state-of-the-art and provides substantial insight into multimodal representation and downstream tasks.
Seeing is Believing: Aligning Prompt Rewriting with Visual Anchors for Text-to-Image Generation
Peking University（北京大学）；Tencent（腾讯）；Dalian University of Technology（大连理工大学）；Nanyang Technological University（南洋理工大学）；University of Cambridge（剑桥大学）；Zhejiang University（浙江大学）
Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift
The Hong Kong Polytechnic University（香港理工大学）；City University of Macau（澳门城市大学）；Jilin University（吉林大学）；Beihang University（北京航空航天大学）；Jinan University（暨南大学）
VizAnchor: Decoding Manipulation Intent from Tampering Visualizations via Dual-Anchor Reasoning
School of Data Science and Engineering, East China Normal University（华东师范大学数据科学与工程学院）；Division of Emerging Interdisciplinary Areas, The Hong Kong University of Science and Technology（香港科技大学新兴跨学科领域分部）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）
SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China（香港城市大学计算机科学系）；Huawei Noah's Ark Lab, Hong Kong SAR, China（华为诺亚实验室（香港））；Huawei's Supply Chain Management Department, Shenzhen, China（华为供应链管理部（深圳））
The Price of Consistency: Exploiting Visual Anchors for Multimodal Jailbreaking in Video Generation
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Hunan University（湖南大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Learning to Understand Body Language from Flight through Robust 3D Avatar Placing
National University of Science and Technology “Politehnica” Bucharest（布加勒斯特理工国立大学）；“Simion Stoilow” Institute of Mathematics of the Romanian Academy（罗马尼亚科学院“西米翁·斯托伊洛夫”数学研究所）；NORCE Norwegian Research Centre AS（挪威NORCE研究中心）
