面向高效具身推理的训练策略
Training Strategies for Efficient Embodied Reasoning
AI总结：
该研究探究机器人思维链推理提升视觉-语言-动作模型性能的内在机制，验证了表示学习等假设，并提出两种轻量推理方案，在LIBERO-90基准上达SOTA且推理速度提升3倍。
摘要：
机器人思维链推理（CoT，即模型在选择动作前预测有用的中间表示）是提升机器人策略（尤其是视觉-语言-动作模型，简称VLA）泛化能力与性能的有效方法。尽管这类方法已被证明可提升性能与泛化性，但存在核心局限，比如需要专用的机器人推理数据，且推理速度较慢。要设计能解决这些问题的新型机器人推理方法，关键在于更全面地厘清推理为何能提升策略性能。我们假设机器人推理提升策略的几种机制：（1）更优的表示学习，（2）改进的学习课程化，（3）更强的表达能力，随后设计了机器人CoT推理的简单变体，以分离并测试每一种机制。我们发现，学习生成推理确实能带来更优的VLA表示，而对推理进行注意力计算则有助于实际利用这些特征来改进动作预测。我们的结果让我们更好地理解了CoT推理为何对VLA有效，基于此我们提出了两种简单轻量的机器人推理替代方案。我们提出的方法相比无推理策略取得了显著性能提升，在LIBERO-90基准上达到了当前最优结果，且推理速度相比标准机器人推理提升了3倍。
英文摘要：
Robot chain-of-thought reasoning (CoT) -- wherein a model predicts helpful intermediate representations before choosing actions -- provides an effective method for improving the generalization and performance of robot policies, especially vision-language-action models (VLAs). While such approaches have been shown to improve performance and generalization, they suffer from core limitations, like needing specialized robot reasoning data and slow inference speeds. To design new robot reasoning approaches that address these issues, a more complete characterization of why reasoning helps policy performance is critical. We hypothesize several mechanisms by which robot reasoning improves policies -- (1) better representation learning, (2) improved learning curricularization, and (3) increased expressivity -- then devise simple variants of robot CoT reasoning to isolate and test each one. We find that learning to generate reasonings does lead to better VLA representations, while attending to the reasonings aids in actually leveraging these features for improved action prediction. Our results provide us with a better understanding of why CoT reasoning helps VLAs, which we use to introduce two simple and lightweight alternative recipes for robot reasoning. Our proposed approaches achieve significant performance gains over non-reasoning policies, state-of-the-art results on the LIBERO-90 benchmark, and a 3x inference speedup compared to standard robot reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
