按需定制的技能：在多种互联网数据引导下高效获取机器人烹饪技能
Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Data
AI总结：
针对从互联网数据学习富接触工具使用技能的挑战，本文提出从一组模板机器人行为中进行选择而非生成低级行为，结合 LLM、视频编码器和光流编码器三种方法，利用多源互联网数据协同作用构建的模板选择器在16项烹饪技能上实现了79%的成功率。
摘要：
本研究探讨了各种互联网数据源的实用性，以在一组模板机器人行为中进行选择来执行技能。由于互联网数据缺乏接触存在性、位置、面积和力等物理信息，从中学习涉及工具使用的富接触技能通常具有挑战性。先前的工作通常使用互联网数据及在此数据上训练的基础模型来生成低级机器人行为。我们假设这些数据和模型可能更适合在一组基本机器人行为中进行选择，以执行这些富接触技能。我们探索了三种模板选择方法：查询大型语言模型；使用先前工作中常见的预训练视频编码器特征，将机器人执行视频与检索到的人类视频进行比较；以及使用在互联网数据上训练的光流编码器特征进行相同比较。我们的结果显示，尽管缺乏视觉信息，LLM 仍是令人惊讶的强大模板选择器；光流编码显著优于使用多一个数量级数据训练的视频编码器；并且各种形式的互联网数据在模板选择之间存在重要的协同作用。通过利用这些协同作用，我们创建了一个使用多种互联网数据的模板选择器，在涉及工具使用的 16 种不同烹饪技能上实现了 79% 的成功率。
英文摘要：
This study explores the utility of various internet data sources to select among a set of template robot behaviors to perform skills. Learning contact-rich skills involving tool use from internet data sources has typically been challenging due to the lack of physical information such as contact existence, location, areas, and force in this data. Prior works have generally used internet data and foundation models trained on this data to generate low-level robot behavior. We hypothesize that these data and models may be better suited to selecting among a set of basic robot behaviors to perform these contact-rich skills. We explore three methods of template selection: querying large language models, comparing video of robot execution to retrieved human video using features from a pretrained video encoder common in prior work, and performing the same comparison using features from an optic flow encoder trained on internet data. Our results show that LLMs are surprisingly capable template selectors despite their lack of visual information, optical flow encoding significantly outperforms video encoders trained with an order of magnitude more data, and important synergies exist between various forms of internet data for template selection. By exploiting these synergies, we create a template selector using multiple forms of internet data that achieves a 79\% success rate on a set of 16 different cooking skills involving tool-use.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
