MAEA：具身智能的多模态归因
MAEA: Multimodal Attribution for Embodied AI
AI总结：
针对具身智能多模态感知理解难题，提出MAEA框架，在ALFRED数据集上解耦视觉、语言和动作输入的全局归因，用于失败分析、偏差调查及部署前鲁棒性评估。
摘要：
理解具身智能（EAI）的多模态感知是一个开放性问题，因为此类输入可能包含高度互补以及冗余的任务相关信息。多模态策略的一个相关方向是理解融合层中各模态的全局趋势。为此，我们在ALFRED数据集上训练的不同策略中，解耦了视觉、语言和先前动作输入的归因。归因分析可用于对失败场景进行排序和分组，调查建模和数据集偏差，并在部署前批判性地分析多模态EAI策略的鲁棒性和用户信任度。我们提出了MAEA，一个用于计算任何可微分策略的每模态全局归因的框架。此外，我们展示了归因如何使EAI策略中语言和视觉归因的较低层级行为分析成为可能。
英文摘要：
Understanding multimodal perception for embodied AI is an open question because such inputs may contain highly complementary as well as redundant information for the task. A relevant direction for multimodal policies is understanding the global trends of each modality at the fusion layer. To this end, we disentangle the attributions for visual, language, and previous action inputs across different policies trained on the ALFRED dataset. Attribution analysis can be utilized to rank and group the failure scenarios, investigate modeling and dataset biases, and critically analyze multimodal EAI policies for robustness and user trust before deployment. We present MAEA, a framework to compute global attributions per modality of any differentiable policy. In addition, we show how attributions enable lower-level behavior analysis in EAI policies for language and visual attributions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
