AI 中文总结
针对多模态意图理解中分歧信息被多数融合方法忽略的问题，提出MACH框架，通过分层一致与冲突原型超图及自适应仲裁机制，在基准数据集上验证了其有效性。
AI 中文摘要
多模态意图识别不仅需要理解文本、声学和视觉信号的共性，还需把握它们的分歧。这类分歧往往包含类别信息；例如，词汇的积极语义若伴随不一致的语音或面部行为，可能暗示讽刺或嘲弄，但大多数融合方法要么鼓励模态对齐，要么将不一致视为需抑制的不确定性。我们提出MACH（模态一致与冲突感知原型超图），这是一种分层原型-超图框架，将多模态一致与冲突表征为不同的、重复出现的关系结构。MACH逐步将单模态表示组合为双模态和三模态抽象。在每个适用层级，模态组合锚点激活稀疏的一致原型超图，以捕捉可复用的共识模式，而独立的冲突路径则将跨模态差异映射至专用的冲突原型超图。两条路径通过逐特征、逐样本自适应仲裁机制结合，使模型能保留有价值的分歧，同时抑制偶然的模态噪声。渐进式优化策略在联合一致-冲突学习前稳定相互依赖的层级结构。在基准数据集上的实验验证了所提方案的有效性，而组件分析与鲁棒性分析则证实了分层组合、原型介导的语义细化及一致-冲突仲裁的独特作用。
英文摘要
Multimodal intent recognition requires understanding not only what textual, acoustic, and visual signals share, but also how they disagree. Such disagreement is frequently class-informative; for example, lexical positivity accompanied by incongruent vocal or facial behavior may indicate sarcasm or taunting, yet most fusion methods either encourage modality alignment or treat inconsistency as uncertainty to be suppressed. We propose MACH (Modality Agreement- and Conflict-aware prototype Hypergraph), a hierarchical prototype-hypergraph framework that represents multimodal agreement and conflict as distinct, recurring relational structures. MACH progressively composes unimodal representations into bimodal and trimodal abstractions. At each applicable level, modality-composition anchors activate sparse agreement prototype hypergraphs that capture reusable consensus patterns, while a separate conflict pathway maps cross-modal discrepancies to dedicated conflict prototype hypergraphs. The two pathways are combined through a feature-wise, sample-adaptive arbitration mechanism, enabling the model to preserve informative disagreement while suppressing incidental modality noise. A progressive optimization strategy stabilizes the interdependent hierarchy before joint agreement-conflict learning. Experiments on benchmark datasets demonstrate the effectiveness of the proposed formulation, while component and robustness analyses validate the distinct roles of hierarchical composition, prototype-mediated semantic refinement, and agreement-conflict arbitration.
