机器人驱动的数据飞轮：在真实世界中部署机器人以实现持续的数据收集和基础模型适应
Robot-Powered Data Flywheels: Deploying Robots in the Wild for Continual Data Collection and Foundation Model Adaptation
AI总结：
机器人驱动的数据飞轮通过部署机器人收集真实世界数据，提升基础模型在复杂环境中的适应能力，减少人工干预并提高多语言OCR性能。
摘要：
基础模型（FM）在视觉和语言领域解锁了强大的零样本能力，但其对互联网预训练数据的依赖使其在无结构的真实世界环境中显得脆弱。在部署过程中遇到的混乱、真实世界的数据（例如遮挡或多语言文本）在现有语料库中严重不足。作为具身代理，机器人 uniquely 有能力填补这一空白：它们可以在物理环境中执行任务，收集大规模的真实世界数据，从而为FM训练提供精确的示例，这些示例是当前模型所缺乏的。我们介绍了机器人驱动的数据飞轮框架，该框架将机器人从FM消费者转变为数据生成者。通过在野外部署配备FM的机器人，我们实现了良性循环：机器人执行有用的任务，同时收集真实世界的数据，从而改进领域特定的适应和领域相邻的泛化。我们通过Scanford，一个在东亚图书馆部署两周的移动机械臂，实例化了这一框架。Scanford自动扫描书架，使用视觉-语言模型（VLM）识别书籍，并利用图书馆目录对图像进行标注，而无需人工标注。这种部署既帮助了图书管理员，也生成了一个用于微调底层VLM的数据集，提高了在特定领域的真实世界图书馆设置中的性能，以及在领域相邻的多语言OCR基准测试中的性能。使用从2103个书架收集的数据，Scanford将VLM在书籍识别上的性能从32.0%提高到71.8%，同时将领域相邻的多语言OCR从24.8%提高到46.6%（英语）和30.8%提高到38.0%（中文），同时节省了约18.7小时的人工时间。这些结果突显了机器人驱动的数据飞轮如何在真实部署中减少人工努力，并解锁了持续适应FM以适应现实混乱的新途径。更多细节请参见：https://scanford-robot.github.io
英文摘要：
Foundation models (FM) have unlocked powerful zero-shot capabilities in vision and language, yet their reliance on internet pretraining data leaves them brittle in unstructured, real-world settings. The messy, real-world data encountered during deployment (e.g. occluded or multilingual text) remains massively underrepresented in existing corpora. Robots, as embodied agents, are uniquely positioned to close this gap: they can act in physical environments to collect large-scale, real-world data that enriches FM training with precisely the examples current models lack. We introduce the Robot-Powered Data Flywheel, a framework that transforms robots from FM consumers into data generators. By deploying robots equipped with FMs in the wild, we enable a virtuous cycle: robots perform useful tasks while collecting real-world data that improves both domain-specific adaptation and domain-adjacent generalization. We instantiate this framework with Scanford, a mobile manipulator deployed in the East Asia Library for 2 weeks. Scanford autonomously scans shelves, identifies books using a vision-language model (VLM), and leverages the library catalog to label images without human annotation. This deployment both aids librarians and produces a dataset to finetune the underlying VLM, improving performance on the domain-specific in-the-wild library setting and on domain-adjacent multilingual OCR benchmarks. Using data collected from 2103 shelves, Scanford improves VLM performance on book identification from 32.0% to 71.8% and boosts domain-adjacent multilingual OCR from 24.8% to 46.6% (English) and 30.8% to 38.0% (Chinese), while saving an ~18.7 hrs of human time. These results highlight how robot-powered data flywheels can both reduce human effort in real deployments and unlock new pathways for continually adapting FMs to the messiness of reality. More details are at: https://scanford-robot.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
