基于基础模型的厨房机器人操作
Kitchen Robotic Manipulation utilizing Foundation Models
查看机构详情
- Kumoh National Institute of Technology（金乌国立技术大学）
- University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本研究提出一种整合多种基础模型的模块化厨房机器人感知流水线，经评估优化后可在杂乱遮挡场景下实现89.12%的ADI，无需环境重训练即可完成餐具处理等家庭操作任务。
中文摘要 AI 辅助
在日常人类环境中部署机器人需要感知系统既具备鲁棒性，又能适应多样、动态的条件。本研究针对厨房环境中的餐具处理任务，提出了一种用于家庭操作任务的模块化感知流水线。该流水线整合了开放词汇目标检测、多视角分割、实例感知三维重建，以及用于六自由度位姿估计与抓取规划的二维-三维特征融合策略。其模块化设计支持系统替换多种视觉与几何基础模型，通过在自定义厨房数据集上的大量评估，我们确定了性能最优的配置。该最优配置（LLMDet + SAMv2 + DINOv2 + GeoTransformer）在包含杂乱与遮挡场景的20场景厨房基准测试中，平均距离积分（ADI）达到89.12%。此外，真实世界演示验证了该最优配置可部署于物理机器人，无需针对特定环境重新训练，成功执行从水槽到洗碗机的物品转移、杯子堆叠等任务。这一结果验证了该流水线的适应性与可扩展性，凸显了其作为家庭机器人系统实用框架的潜力。我们的代码及补充材料可在此httpsURL获取。
英文摘要
Deploying robots in everyday human environments requires perception systems that are both robust and adaptable to diverse, dynamic conditions. In this work, we present a modular perception pipeline for household manipulation tasks, with a focus on dishware handling in kitchen environments. The pipeline integrates open-vocabulary object detection, multi-view segmentation, instance-aware 3D reconstruction, and a 2D-3D feature fusion strategy for 6D pose estimation and grasp planning. Its modular design enables systematic substitution of multiple visual and geometric foundation models, allowing us to identify the best-performing configuration through extensive evaluation on a custom kitchen dataset. The best-performing configuration (LLMDet + SAMv2 + DINOv2 + GeoTransformer) achieves an ADI of 89.12\% on the 20-scene kitchen benchmark with cluttered and occluded conditions. Furthermore, real-world demonstrations confirm that the best configuration can be deployed on physical robots without environment-specific retraining, successfully executing tasks such as sink-to-dishwasher transfer and cup stacking. It validates the adaptability and scalability of the pipeline and highlights its potential as a practical framework for household robotic systems. Our code and supplementary materials are available at https://raivlab.github.io/FM_kitchen .
