Affogato: 基于大规模自动数据生成的开词汇可操作区域定位
Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale
AI总结：
提出Affogato框架，通过自动化流程生成750K 3D可操作区域热力图与自然语言查询对，解决数据瓶颈，实现开词汇可操作区域定位，并验证其跨架构迁移能力。
摘要：
可操作区域定位旨在定位与物体交互的位置，这是具身智能体的基本能力。然而，进展受到数据瓶颈的限制：人工标注成本高昂，且现有数据集局限于少量预定义的物体和可操作区域类别。我们提出Affogato，一个以Affogato-750K为核心的开词汇可操作区域定位框架，该数据集包含75万个3D可操作区域热力图及其对应的自然语言查询。我们通过全自动流程构建该数据集，协调基础模型在无需人工标注的情况下大规模生成数据。其覆盖的类别多样性远超现有数据集。为进行可靠评估，我们还提供了5千个人工验证的测试对。此外，我们提出了Espresso-3D和Espresso-2D，这两种简单而有效的模型采用跨模态的统一架构。在Affogato-750K上的预训练提升了Espresso及先前方法的性能，并在未见过的物体和可操作区域类别上取得了最大增益，表明其提供了跨架构的广泛可迁移监督。
英文摘要：
Affordance grounding aims to localize where to interact with an object, a fundamental capability for embodied agents. Yet progress is bottlenecked by data: manual annotation is prohibitively expensive and confines existing datasets to a narrow set of predefined object and affordance categories. We introduce Affogato, a framework for open-vocabulary affordance grounding centered on Affogato-750K, a large-scale dataset of 750K 3D affordance heatmaps paired with natural language queries. We build it with a fully automated pipeline that orchestrates foundation models to generate them at scale without human labeling. It covers significantly more diverse categories than any existing dataset. For reliable evaluation, we further provide 5K human-verified test pairs. We also present Espresso-3D and Espresso-2D, simple yet effective models with a unified architecture across both modalities. Pretraining on Affogato-750K improves both Espresso and prior methods and yields the largest gains on unseen object and affordance categories, showing that it provides broadly transferable supervision across architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
