AnnotateAnything：面向机器人操作的3D资产自动标注
AnnotateAnything: Automatic Annotation of 3D Assets for Robot Manipulation
AI总结：
提出AnnotateAnything框架，通过视觉-语言标注和物理标注双流水线，自动为3D资产生成可执行操作标签，提升仿真数据收集效率和任务成功率。
摘要：
仿真使得可扩展的机器人数据收集成为可能，但原始3D资产仅提供几何信息，缺乏指定机器人应在何处以及如何操作的语义、交互和物理知识。在这项工作中，我们提出了AnnotateAnything，一个通用的自动标注框架，将被动3D资产转换为具有结构化、多样化和可执行操作标签的、可用于操作的资产。AnnotateAnything围绕两个互补的流水线构建。首先，一个统一的视觉-语言标注流水线，利用视觉-语言推理来推断对象语义、交互约束和3D接地线索，为识别有意义的交互区域提供人类先验指导。其次，一个全自动且大规模并行的物理标注流水线，通过候选生成、几何优化和轨迹生成，将这些先验知识嵌入每个资产的几何和物理约束中。该流水线生成多样且可执行的动作标注，包括抓取姿态、灵巧接触、关节运动路径点、插入方向、悬挂可供性和导航目标。利用生成的标注，我们进一步构建了一个跨不同对象、任务和机器人形态的异步并行仿真数据收集系统。实验表明，与现有的标注和数据生成流水线相比，AnnotateAnything在标注效率、数据收集效率和任务成功率方面均表现优越，同时支持下游任务如可供性检测、机器人VQA和视觉指令微调。我们在项目页面上提供项目材料，并计划发布完整代码、标注和基准以促进未来研究。视频、代码、演示资产和标注在补充材料中提供。项目页面：此https URL。
英文摘要：
Simulation enables scalable robot data collection, but raw 3D assets provide only geometry, lacking the semantic, interactive, and physical knowledge needed to specify where and how robots should act. In this work, we present AnnotateAnything, a general automatic annotation framework that converts passive 3D assets into manipulation-ready assets with structured, diverse, and executable manipulation labels. AnnotateAnything is built around two complementary pipelines. First, a unified visual-language annotation pipeline using vision-language reasoning to infer object semantics, interaction constraints, and 3D-grounded cues, providing human-prior guidance for identifying meaningful interaction regions. Second, a fully automatic and massively parallel physics annotation pipeline grounds these priors in each asset's geometry and physical constraints through candidate generation, geometry optimization and trajectory generation. This pipeline produces diverse and executable action annotations, including grasp poses, dexterous contacts, articulation waypoints, insertion directions, hanging affordances, and navigation targets. Using the generated annotations, we further build an asynchronous parallel simulation data-collection system across diverse objects, tasks, and robot embodiments. Experiments demonstrate that AnnotateAnything achieves superior annotation efficiency, data-collection efficiency, and task success rates over existing annotation and data-generation pipelines, while also supporting downstream tasks such as affordance detection, robotic VQA, and visual instruction finetuning. We provide project materials on the project page and plan to release the full code, annotations, and benchmark to facilitate future research. Videos, code, demo assets, and annotations are provided in supplementary materials Project page: https://tourmaline-caramel-169490.netlify.app.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
