面向生成式技能获取的具身推理实时验证
Real-Time Verification of Embodied Reasoning for Generative Skill Acquisition
AI总结：
针对现有生成式技能获取方法在3D环境中效率低、奖励工程繁琐的问题，提出VERGSA框架，将实时验证融入具身技能学习，构建相关训练数据集，显著提升任务成功率与验证质量。
摘要：
生成式技能获取使具身智能体能够主动学习一套可扩展、不断演进的控制技能库，这对大型决策模型的发展至关重要。现有方法通常依赖通才智能体（如大语言模型LLM）的监督信号，但其在复杂3D环境中的有效性仍不明确；穷尽式评估会产生巨大计算成本，严重阻碍了技能学习的效率。受数学推理验证模型近期取得的成功启发，我们提出VERGSA（生成式技能获取中的具身推理验证）框架，将实时验证原理系统性地融入具身技能学习过程。VERGSA实现了两大核心设计：1）通过在提示词中动态纳入上下文相关任务，并为子任务和整体任务分别定义成功指标，将数学推理验证方法无缝扩展到具身学习领域；2）提出一种自动化、可扩展的奖励标注方案，通过迭代确定场景配置和子任务学习对整体技能获取的贡献，合成密集奖励信号。据我们所知，该方法构建了首个面向验证驱动的生成式技能获取的综合训练数据集，免去了繁琐的人工奖励工程。实验验证了我们方法的有效性：1）示例任务池将平均任务成功率提升了21%；2）我们的验证模型将新任务的成功率提升了24%，将已遇任务的成功率提升了36%；3）在验证质量上优于LLM-as-a-Judge基线。
英文摘要：
Generative skill acquisition enables embodied agents to actively learn a scalable and evolving repertoire of control skills, crucial for the advancement of large decision models. While prior approaches often rely on supervision signals from generalist agents (e.g., LLMs), their effectiveness in complex 3D environments remains unclear; exhaustive evaluation incurs substantial computational costs, significantly hindering the efficiency of skill learning. Inspired by recent successes in verification models for mathematical reasoning, we propose VERGSA (Verifying Embodied Reasoning in Generative Skill Acquisition), a framework that systematically integrates real-time verification principles into embodied skill learning. VERGSA establishes 1) a seamless extension from verification of mathematical reasoning into embodied learning by dynamically incorporating contextually relevant tasks into prompts and defining success metrics for both subtasks and overall tasks, and 2) an automated, scalable reward labeling scheme that synthesizes dense reward signals by iteratively finalizing the contribution of scene configuration and subtask learning to overall skill acquisition. To the best of our knowledge, this approach constitutes the first comprehensive training dataset for verification-driven generative skill acquisition, eliminating arduous manual reward engineering. Experiments validate the efficacy of our approach: 1) the exemplar task pool improves the average task success rates by 21%, 2) our verification model boosts success rates by 24% for novel tasks and 36% for encountered tasks, and 3) outperforms LLM-as-a-Judge baselines in verification quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
