arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.20351cs.ROcs.AI

作为任意机器人后训练数据的人类演示：重定位对象而非手部

Human Demonstrations as Post-Training Data for Any Robot Body: Retargeting the Object, Not the Hand

Dxtr AI Co.（Dxtr AI公司）

Nagasrikanth Kallakuri

备注：
16 pages, 17 figures, 7 tables, 2 algorithms; 56-page supplement as ancillary file. A subset of episodes with their per-robot outcomes: https://playground.dxtr.co
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究提出将人类演示作为任意机器人的后训练数据，通过重定位对象而非手部实现，在8个机器人的157个对中，31.2%通过，弃权（不执行）的对若执行则全部失败。

摘要：

如今，人类演示主要用于预训练视觉-语言-动作模型，几乎无法将相同数据用于目标体或面向任务的训练，而机器人在工业及通用环境中的可扩展部署正依赖于这类训练。通过手机等简单且可扩展设备记录的人类演示，可作为身体结构和能力与演示者不同的机器人的后训练数据，这一实现并非通过重定位手部运动，而是通过重定位对象。该捕获过程会生成一份无身体的记录，包含对象随时间的位姿、接触间隔以及带溯源信息的重建3D对象。给定URDF（统一机器人描述格式）和简短描述符的任意机器人，会针对该记录求解，并在仿真中以演示的对象运动为基准进行评分，这是“人类-仿真-机器人”迁移中的“人类到仿真”环节，按每个机器人单独测量。每个（机器人， episodes）对会在任何运动前获得一个结果：通过、带命名模式的失败或弃权（不执行）并附带原因。该方法作为一个工作系统运行在不断增长的集合上，本文分析的集合是其样本：在8个机器人上求解的42次捕获，共157个对，其中31.2%通过，52.2%失败，14.0%弃权（不执行），2.5%不可审计（仿真中运动学可行性，±50毫米/±15度）。弃权（不执行）的情况经检验：所有检查弃权的对，若仍执行则全部失败（13对中的13对；5对的机器人其公开模型将手部焊接）。无法证明描述文件可筛选机器人：其结果几乎不比随机猜测更可预测。目前尚未基于该记录训练任何策略。该记录决定机器人可在哪些内容上进行后训练；而策略是否能在真实机器人上生效，属于“仿真到机器人”的问题，将在下游环节解答。

英文摘要：

Human demonstrations are today used largely for pretraining vision-language-action models, with little ability to use the same data for target-embodiment or task-oriented training, which scalable deployment of robots in industrial and general environments depends on. A human demonstration recorded on a simple and scalable device such as a mobile phone can serve as post-training data for a robot whose body differs in structure and capability from the human who recorded it. This is made possible not by retargeting the hand's motion but the object's. A capture yields a body-free record: the object's pose over time, its contact intervals, and a reconstructed 3D object with provenance. Any robot, given as a URDF and a short descriptor, is solved against that record and scored in simulation against the demonstrated object motion as ground truth: the human-to-simulation leg of human-to-simulation-to-robot transfer, measured per robot. Each (robot, episode) pair gets one outcome: pass, failure with a named mode, or refusal, with its reason, before any motion. The method runs as a working system over a growing collection; the set analysed here is a sample of it: 42 captures solved on 8 robots, 157 pairs, of which 31.2% pass, 52.2% fail, 14.0% are refused and 2.5% are unauditable (kinematic feasibility, +/-50 mm / +/-15 deg, in simulation). Refusals held up: every pair the checks refuse on stored evidence yet ran anyway failed (13 of 13; 5 on a body whose public model welds its hand). A description file could not be shown to screen a robot: its outcomes were barely more predictable than guessing. No policy has yet been trained on the record. The record decides what a robot may post-train on; whether the policy then works on the real robot is the simulation-to-robot question, answered downstream.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑