展示而非告知：通过观看人类视频检测新物体
Show, Don't Tell: Detecting Novel Objects by Watching Human Videos
AI总结：
本文提出一种自监督系统，通过自动创建的数据集训练定制物体检测器，无需复杂语言描述和昂贵的提示工程，从而快速训练适合人类任务演示中观察到的特定物体的检测器。
摘要：
如何让机器人快速识别和识别在人类演示中展示的新物体？现有封闭集物体检测器往往失败，因为这些物体是分布外的。虽然开放集检测器（如VLMs）有时能成功，但通常需要昂贵且繁琐的人工在环提示工程来唯一识别新物体实例。在本文中，我们提出了一种自监督系统，通过训练一个定制的物体检测器在自动创建的数据集上进行监督，该数据集由人类演示本身监督。在我们的方法中，“展示而非告知”，我们向检测器展示感兴趣的特定物体，而不是通过复杂的语言描述告诉检测器这些物体。通过完全绕过语言，这种范式使我们能够快速训练针对人类任务演示中观察到的相关物体的定制检测器。我们开发了一个集成的机器人系统，用于部署我们的“展示而非告知”范式，即自动数据集创建和新物体检测在真实世界机器人上。实证结果表明，我们的流水线在 manipulated objects 的检测和识别方法上显著优于最先进的方法，从而提高机器人的任务完成能力。
英文摘要：
How can a robot quickly identify and recognize new objects shown to it during a human demonstration? Existing closed-set object detectors frequently fail at this because the objects are out-of-distribution. While open-set detectors (e.g., VLMs) sometimes succeed, they often require expensive and tedious human-in-the-loop prompt engineering to uniquely recognize novel object instances. In this paper, we present a self-supervised system that eliminates the need for tedious language descriptions and expensive prompt engineering by training a bespoke object detector on an automatically created dataset, supervised by the human demonstration itself. In our approach, "Show, Don't Tell," we show the detector the specific objects of interest during the demonstration, rather than telling the detector about these objects via complex language descriptions. By bypassing language altogether, this paradigm enables us to quickly train bespoke detectors tailored to the relevant objects observed in human task demonstrations. We develop an integrated on-robot system to deploy our "Show, Don't Tell" paradigm of automatic dataset creation and novel object-detection on a real-world robot. Empirical results demonstrate that our pipeline significantly outperforms state-of-the-art detection and recognition methods for manipulated objects, leading to improved task completion for the robot.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
