arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.20072cs.AIcs.RO
收藏

探查具身大语言模型:当更高的观察保真度损害问题解决

Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving

Robotics and Biology Laboratory, Technische Universität Berlin, Germany(柏林技术大学机器人与生物学实验室);Science of Intelligence, Research Cluster of Excellence, Berlin, Germany(柏林科学智能研究卓越集群);Robotics Institute Germany (RIG)(德国机器人研究所(RIG))

Oussama Zenkri, Oliver Brock

AI总结:

本文研究了具身大语言模型在不同观察信息下的行为,发现高保真度观察反而降低了问题解决能力,核心方法是通过实验改变可用信息并测量行为变化,主要贡献是揭示了感知误差与推理失败的交互影响。

摘要:

大型语言模型日益被提出作为机器人系统的认知组件,但其不透明的决策过程使得在闭环具身任务中的成功或失败难以解释。遵循经验AI方法,我们通过改变代理可用的信息并测量行为变化来研究具身LLM代理的行为。使用Lockbox,一个具有隐藏依赖关系的顺序机械谜题,在物理机器人设置中评估LLM在RGB、RGB-D和地面真实符号观察下的表现,并通过受控模拟来探测由此产生行为。反直觉的是,代理在原始RGB输入下表现最佳,而在完美地面真实观察下表现最差。在模拟中,我们通过随机翻转感知的动作结果来探测这一效应,发现适度的噪声提高了性能,峰值出现在40%的翻转概率下,相比无噪声基线,成功率提高了2.85倍。进一步分析将这一收益归因于重复动作循环的减少。这些发现表明,仅凭成功率来评估LLM是不够的,因为测量性能可能反映了感知误差与推理失败之间的相互作用,而非稳健的问题解决。

英文摘要:

Large Language Models (LLMs) are increasingly proposed as cognitive components for robotic systems, yet their opaque decision processes make it difficult to explain success or failure in closed-loop embodied tasks. Following an empirical AI methodology, we study an embodied LLM agent behaviorally by varying the available information and measuring the resulting changes in behavior. Using the Lockbox, a sequential mechanical puzzle with hidden interdependencies, we evaluate LLMs across RGB, RGB-D, and ground-truth symbolic observations in a physical robotic setup and use simulation to probe the resulting behavior. Counterintuitively, agents perform best under raw RGB input and worst under perfect ground-truth observations. In simulation, we probe this effect by randomly flipping perceived action outcomes and find that moderate noise improves performance, peaking at a 40% flip probability with a 2.85-fold success rate increase over the noise-free baseline. Further analysis links this gain to a reduction in repetitive action loops. These findings suggest that success rates alone are insufficient for evaluating LLMs, as measured performance may reflect the interaction between perceptual errors and reasoning failures rather than robust problem solving.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑