Video-to-BT：从人类演示视频生成用于机器人装配的反应式行为树
Video-to-BT: Generating Reactive Behavior Trees from Human Demonstration Videos for Robotic Assembly
AI总结：
本文提出 Video-to-BT 分层框架，利用视觉语言模型从人类演示视频生成行为树，结合实时场景解释与失败重规划，实现机器人装配在动态环境中的反应式执行和强泛化能力。
摘要：
现代制造业要求机器人装配系统具备更高的灵活性和可靠性。然而，传统方法通常依赖专家针对固定设置对每个产品进行编程，这本质上对产品变更缺乏灵活性，并且缺乏处理变化的鲁棒性。随着行为树（BTs）因其模块化和反应性而在机器人领域得到越来越广泛的应用，我们提出了一种新颖的分层框架 Video-to-BT，它将高层认知规划与低层反应式控制无缝集成，其中行为树既作为规划的结构化输出，也作为执行的治理结构。我们的方法利用视觉语言模型（VLM）将人类演示视频分解为子任务，并据此生成行为树。在执行过程中，规划好的行为树与实时场景解释相结合，使系统能够在动态环境中以反应式方式运行，同时在执行失败时触发由 VLM 驱动的重新规划。这种闭环架构确保了稳定性和适应性。我们通过一系列实验在真实世界的装配任务上验证了该框架，展示了较高的规划可靠性、在长时程装配任务中的鲁棒性能，以及在多样化和扰动条件下的强泛化能力。项目网站：https://video2bt.github.io/video2bt_page/
英文摘要：
Modern manufacturing demands robotic assembly systems with enhanced flexibility and reliability. However, traditional approaches often rely on programming tailored to each product by experts for fixed settings, which are inherently inflexible to product changes and lack the robustness to handle variations. As Behavior Trees (BTs) are increasingly used in robotics for their modularity and reactivity, we propose a novel hierarchical framework, Video-to-BT, that seamlessly integrates high-level cognitive planning with low-level reactive control, with BTs serving both as the structured output of planning and as the governing structure for execution. Our approach leverages a Vision-Language Model (VLM) to decompose human demonstration videos into subtasks, from which Behavior Trees are generated. During the execution, the planned BTs combined with real-time scene interpretation enable the system to operate reactively in the dynamic environment, while VLM-driven replanning is triggered upon execution failure. This closed-loop architecture ensures stability and adaptivity. We validate our framework on real-world assembly tasks through a series of experiments, demonstrating high planning reliability, robust performance in long-horizon assembly tasks, and strong generalization across diverse and perturbed conditions. Project website: https://video2bt.github.io/video2bt_page/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
