探究指令多样性与任务难度在机器人操作任务中的作用
Investigating the Role of Instruction Variety and Task Difficulty in Robotic Manipulation Tasks
AI总结：
本文提出综合评估框架，从架构、跨模态输入扰动和任务复杂度考察机器人操作多模态模型的泛化性，发现其抗指令扰动但易受观测变化影响，指出需改进架构与训练。
摘要：
仅依据多模态模型在分布外数据上的表现来评估其泛化能力，无法反映其真实鲁棒性。本研究提出一个综合评估框架，系统考察指令和输入在此类模型泛化能力中的作用，同时考虑架构设计、跨语言与视觉模态的输入扰动，以及不断提升的任务复杂度。该框架揭示了多模态模型对极端指令扰动的韧性，以及它们对观测变化的脆弱性，引发了对模型过拟合虚假相关性的担忧。通过将此评估框架应用于当前用于机器人操作任务的基于 Transformer 的多模态模型，我们揭示了其局限性，并建议未来进展应聚焦于能更好整合多模态输入的架构与训练创新，通过优先关注输入内容而非偶然相关性来增强模型的泛化能力。
英文摘要：
Evaluating the generalisation capabilities of multimodal models based solely on their performance on out-of-distribution data fails to capture their true robustness. This work introduces a comprehensive evaluation framework that systematically examines the role of instructions and inputs in the generalisation abilities of such models, considering architectural design, input perturbations across language and vision modalities, and increased task complexity. The proposed framework uncovers the resilience of multimodal models to extreme instruction perturbations and their vulnerability to observational changes, raising concerns about overfitting to spurious correlations. By employing this evaluation framework on current Transformer-based multimodal models for robotic manipulation tasks, we uncover limitations and suggest future advancements should focus on architectural and training innovations that better integrate multimodal inputs, enhancing a model's generalisation prowess by prioritising sensitivity to input content over incidental correlations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
