T-Rex：面向视觉语言模型机器人操作的任务自适应空间表征提取
T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Models
AI总结：
针对现有视觉语言模型机器人操作中固定空间表征提取方案的不足，提出任务自适应框架T-Rex，动态选择最优提取方案，在真实环境中无需训练即提升空间理解、效率与稳定性。
摘要：
构建一个能够在真实世界环境中执行各种任务的通用机器人操作系统是一项具有挑战性的任务。视觉语言模型（VLMs）在机器人操作任务中展现出了巨大的潜力，这主要归功于它们从大规模数据集中获得的大量世界知识。在此过程中，空间表征（如表示物体位置的点或表示物体方向的向量）充当了视觉语言模型与真实世界场景之间的桥梁，有效地将视觉语言模型的推理能力落地并应用于特定任务场景。然而，现有的基于视觉语言模型的机器人方法通常对各种任务采用固定的空间表征提取方案，导致表征能力不足或提取时间过长。在这项工作中，我们提出了T-Rex，一个任务自适应空间表征提取框架，它根据具体任务需求动态地为每个实体选择最合适的空间表征提取方案。我们的关键见解是，任务复杂性决定了空间表征的类型和粒度，而更强的表征能力通常与更高的整体系统运行成本相关联。通过在真实机器人环境中的全面实验，我们表明我们的方法在无需额外训练的情况下，在空间理解、效率和稳定性方面带来了显著优势。
英文摘要：
Building a general robotic manipulation system capable of performing a wide variety of tasks in real-world settings is a challenging task. Vision-Language Models (VLMs) have demonstrated remarkable potential in robotic manipulation tasks, primarily due to the extensive world knowledge they gain from large-scale datasets. In this process, Spatial Representations (such as points representing object positions or vectors representing object orientations) act as a bridge between VLMs and real-world scene, effectively grounding the reasoning abilities of VLMs and applying them to specific task scenarios. However, existing VLM-based robotic approaches often adopt a fixed spatial representation extraction scheme for various tasks, resulting in insufficient representational capability or excessive extraction time. In this work, we introduce T-Rex, a Task-Adaptive Framework for Spatial Representation Extraction, which dynamically selects the most appropriate spatial representation extraction scheme for each entity based on specific task requirements. Our key insight is that task complexity determines the types and granularity of spatial representations, and Stronger representational capabilities are typically associated with Higher overall system operation costs. Through comprehensive experiments in real-world robotic environments, we show that our approach delivers significant advantages in spatial understanding, efficiency, and stability without additional training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
