当意义旅行：混合专家模型在语言模型习语理解中的细粒度作用
When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models
AI总结：
针对低资源东南亚语言中习语的文化隐喻复杂性，提出Varnika多模态习语语料库和混合专家模型HybridMoE，通过控制混合和掩码多模态嵌入缓解专家稀疏性，实现习语理解性能提升5-6%。
摘要：
在多语言教育的当代，学习习语为通往创造力、文化价值观、历史背景以及各种语言传统固有的多元视角提供了迷人的途径。本文展示了在低资源东南亚语言（如印地语、孟加拉语和泰语）中保留比喻和文化语义的导航，这些语言中文化丰富的习语由于其深刻的隐喻复杂性，给计算建模和跨语言迁移带来了重大障碍。为应对这种复杂性，我们提出了Varnika，一个重建的多模态习语语料库，包含3,533个多语言习语，并丰富了与文本和视觉表示对齐的七种习语语气。此外，为了推断信息丰富的习语理解，我们引入了一个混合专家模型（HybridMoE）框架，该框架嵌入多个习语专家意见，同时通过受控混合整合来自选定和未选定专家的输出来缓解专家稀疏性，并通过掩码多模态嵌入进一步增强了习语属性信号。为了跨多个维度分析性能，我们提出了IDIO-TONE和习语验证分数，这是一个三阶段评估流程，衡量（i）字面翻译保真度，（ii）视觉语义对齐，以及（iii）习语意义保留。实证评估表明，HybridMoE在先进的视觉语言模型上实现了5-6%的性能提升，展示了在多语言多模态设置中比喻语言和文化嵌入意义的改进表示。
英文摘要：
In the contemporary epoch of multilingual education, learning idioms provides a fascinating gateway towards creativity, cultural values, historical context, and diverse perspectives inherent to various linguistic traditions. This paper showcases the navigation of retaining figurative and cultural semantics in low-resource Southeast Asian languages such as Hindi, Bengali, and Thai, where culturally rich idioms pose significant obstacles for computational modeling and cross-linguistic transfer due to their deep metaphorical complexity. To tackle such complexity, we present Varnika, a reconstructed multimodal idiom corpus comprising 3,533 multilingual idioms, enriched with seven idiomatic tones aligned with both textual and visual representations. Additionally, to infer informative idiomatic understanding, we introduce a Hybrid Mixture-of-Experts (HybridMoE) framework that embeds multiple idiomatic expert opinions while mitigating expert sparsity by integrating outputs from both selected and unselected experts through controlled hybridization, further augmented with Idiomatic Property Signals via masked multimodal embeddings. To analyze the performance across multiple dimensions, we propose the IDIO-TONE and Idiomatic Validation Score, a three-stage evaluation pipeline measuring (i) literal translation fidelity, (ii) visual-semantic alignment, and (iii) idiomatic meaning retention. Empirical evaluations highlight that HybridMoE achieves 5--6\% performance gains across advanced vision language models, demonstrating improved representation of figurative language and culturally embedded meaning in multilingual multimodal settings
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
