MOSAIC：用于辅助和交互式烹饪的模块化基础模型
MOSAIC: Modular Foundation Models for Assistive and Interactive Cooking
AI总结：
MOSAIC提出模块化架构，结合预训练基础模型与专用控制模块，实现多机器人自然语言交互和开放词汇物品操作，经单元测试和端到端试验验证，可扩展至复杂人机协作任务。
摘要：
我们提出了MOSAIC，一种用于协调多个机器人的模块化架构，使其能够（a）使用自然语言与用户交互，以及（b）操作开放词汇表中的日常物品。MOSAIC在多个层面采用模块化设计：它利用多个大规模预训练模型处理语言和图像识别等高层任务，同时使用为低层任务特定控制设计的精简模块。这种分解使我们能够兼得基础模型与精确、更专门化模型的互补优势。组合在一起，我们的系统能够扩展到涉及协调多个机器人和人类的复杂任务。首先，我们通过180次视觉运动抓取、60次人体运动预测和46次在线用户对任务规划器的评估对各个模块进行单元测试。然后，我们用60次端到端试验对MOSAIC进行了广泛评估。我们讨论了关键的设计决策、当前系统的局限性以及该领域的开放挑战。项目网站位于https://portal-cornell.github.io/MOSAIC/。
英文摘要：
We present MOSAIC, a modular architecture for coordinating multiple robots to (a) interact with users using natural language and (b) manipulate an open vocabulary of everyday objects. MOSAIC employs modularity at several levels: it leverages multiple large-scale pre-trained models for high-level tasks like language and image recognition, while using streamlined modules designed for low-level task-specific control. This decomposition allows us to reap the complementary benefits of foundation models as well as precise, more specialized models. Pieced together, our system is able to scale to complex tasks that involve coordinating multiple robots and humans. First, we unit-test individual modules with 180 episodes of visuomotor picking, 60 episodes of human motion forecasting, and 46 online user evaluations of the task planner. We then extensively evaluate MOSAIC with 60 end-to-end trials. We discuss crucial design decisions, limitations of the current system, and open challenges in this domain. The project's website is at https://portal-cornell.github.io/MOSAIC/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
