ForceSight：基于视觉-力目标的文本引导移动操作
ForceSight: Text-Guided Mobile Manipulation with Visual-Force Goals
AI总结：
提出 ForceSight，利用 RGBD 图像与文本提示预测运动学位姿和力组成的视觉-力目标，显著提升移动操作机器人在未知环境中的任务成功率。
摘要：
我们提出了 ForceSight，这是一个用于文本引导移动操作的系统，利用深度神经网络预测视觉-力目标。给定单张 RGBD 图像并结合文本提示，ForceSight 可确定相机坐标系下的目标末端执行器位姿（运动学目标）以及相关的力（力目标）。这两个组成部分共同构成一个视觉-力目标。先前工作已表明，输出人类可解释运动学目标的深度模型能够使真实机器人实现灵巧操作。力对操作至关重要，但在此类系统中通常被置于较低层的执行环节。当部署在配备眼在手上 RGBD 相机的移动操作机器人上时，ForceSight 在未见过的环境中执行精密抓取、抽屉开启和物体交接等任务，成功率达到 81%，且这些环境中的物体实例与训练数据存在显著差异。在另一项独立实验中，仅依赖视觉伺服而忽略力目标会使成功率从 90% 降至 45%，证明力目标能够显著提升性能。附录、视频、代码和训练模型可在 https://force-sight.github.io/ 获取。
英文摘要：
We present ForceSight, a system for text-guided mobile manipulation that predicts visual-force goals using a deep neural network. Given a single RGBD image combined with a text prompt, ForceSight determines a target end-effector pose in the camera frame (kinematic goal) and the associated forces (force goal). Together, these two components form a visual-force goal. Prior work has demonstrated that deep models outputting human-interpretable kinematic goals can enable dexterous manipulation by real robots. Forces are critical to manipulation, yet have typically been relegated to lower-level execution in these systems. When deployed on a mobile manipulator equipped with an eye-in-hand RGBD camera, ForceSight performed tasks such as precision grasps, drawer opening, and object handovers with an 81% success rate in unseen environments with object instances that differed significantly from the training data. In a separate experiment, relying exclusively on visual servoing and ignoring force goals dropped the success rate from 90% to 45%, demonstrating that force goals can significantly enhance performance. The appendix, videos, code, and trained models are available at https://force-sight.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
