即插即用澄清器：一种用于第一人称意图消歧的零样本多模态框架
Plug-and-Play Clarifier: A Zero-Shot Multimodal Framework for Egocentric Intent Disambiguation
AI总结：
针对第一人称AI智能体面临的多模态意图模糊问题，提出零样本模块化即插即用澄清器框架，通过三个协同模块分解任务，提升小模型意图澄清性能约30%，并增强大型模型效果及视觉、跨模态子任务准确率，优化用户体验。
摘要：
第一人称AI智能体的性能从根本上受到多模态意图模糊性的限制。这一挑战源于语言描述不明确、视觉数据不完善以及指示性手势的共同作用，这些因素经常导致任务失败。现有的单一体视觉语言模型（VLMs）难以处理这些多模态模糊输入，常常无声地失败或生成幻觉响应。为解决这些模糊性问题，我们引入了即插即用澄清器（Plug-and-Play Clarifier），这是一种零样本模块化框架，将问题分解为离散、可解决的子任务。具体而言，我们的框架包含三个协同模块：（1）文本澄清器，利用对话驱动推理以交互方式消除语言意图的模糊性；（2）视觉澄清器，提供实时指导反馈，指示用户调整位置以提高捕捉质量；（3）带接地机制的跨模态澄清器，可稳健地解释3D指向手势并识别用户指向的特定对象。大量实验表明，我们的框架将小型语言模型（4--8B）的意图澄清性能提高了约30%，使其能与显著更大的模型相竞争。我们还观察到将框架应用于这些大型模型时也能获得持续收益。此外，我们的视觉澄清器将纠正性指导准确率提高了20%以上，跨模态澄清器将指称接地的语义答案准确率提高了5%。总体而言，我们的方法提供了一个即插即用框架，能有效解决多模态模糊性并显著提升第一人称交互中的用户体验。
英文摘要：
The performance of egocentric AI agents is fundamentally limited by multimodal intent ambiguity. This challenge arises from a combination of underspecified language, imperfect visual data, and deictic gestures, which frequently leads to task failure. Existing monolithic Vision-Language Models (VLMs) struggle to resolve these multimodal ambiguous inputs, often failing silently or hallucinating responses. To address these ambiguities, we introduce the Plug-and-Play Clarifier, a zero-shot and modular framework that decomposes the problem into discrete, solvable sub-tasks. Specifically, our framework consists of three synergistic modules: (1) a text clarifier that uses dialogue-driven reasoning to interactively disambiguate linguistic intent, (2) a vision clarifier that delivers real-time guidance feedback, instructing users to adjust their positioning for improved capture quality, and (3) a cross-modal clarifier with grounding mechanism that robustly interprets 3D pointing gestures and identifies the specific objects users are pointing to. Extensive experiments demonstrate that our framework improves the intent clarification performance of small language models (4--8B) by approximately 30%, making them competitive with significantly larger counterparts. We also observe consistent gains when applying our framework to these larger models. Furthermore, our vision clarifier increases corrective guidance accuracy by over 20%, and our cross-modal clarifier improves semantic answer accuracy for referential grounding by 5%. Overall, our method provides a plug-and-play framework that effectively resolves multimodal ambiguity and significantly enhances user experience in egocentric interaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
