从局部全身VLA行为到场景级空中操控

中文摘要 AI 辅助

视觉-语言-动作（VLA）模型支持任务条件下的交互，但将其扩展到场景级空中操控仍面临挑战，原因在于昂贵的全身演示、延迟引起的动作-状态错位以及跨站点行为组合。我们提出一个统一框架，用于在关节式无人空中操控器（UAM）上进行合成策略训练和场景级执行。一个场景可重构的流程合成任务条件、运动动力学可行轨迹以及同步多视角观测，用于VLA训练，无需物理平台演示。测量进度对齐实现（MPAR）将异步返回的动作块与测量执行进度对齐，并将其实现为连续、动态可行的轨迹。一个关系场景图将语言目标锚定到对象实例和可行的交互区域，而拓扑引导转移连接各站点间的局部行为。在oracle目标和可行交接条件下，局部VLA技能在模拟中实现39/60次成功（65.0%）。在500毫秒添加延迟下，有或无瞬时命令更新停顿，MPAR将中位接管相位误差比标称时间对齐降低0.212秒。完整系统完成21/50次模拟多站点任务（42.0%），并在物理关节式UAM上进一步验证。

英文摘要

Vision-language-action (VLA) models enable task-conditioned interaction, but extending them to scene-scale aerial manipulation remains challenging due to costly whole-body demonstrations, latency-induced action-state misalignment, and cross-site behavior composition. We present a unified framework for synthetic policy training and scene-scale execution on articulated uncrewed aerial manipulators (UAMs). A scene-reconfigurable pipeline synthesizes task-conditioned, kinodynamically feasible trajectories and synchronized multiview observations for VLA training without physical-platform demonstrations. Measured-progress-aligned realization (MPAR) aligns asynchronously returned action chunks with measured execution progress and realizes them as continuous, dynamically feasible trajectories. A relational Scene Graph grounds language goals to object instances and feasible interaction regions, while topology-guided transfer connects local behaviors across sites. Local VLA skills achieve 39/60 successes (65.0%) in simulation under oracle target and feasible-handoff conditions. Under 500-ms added latency, with and without a transient command-update stall, MPAR reduces median takeover phase error by 0.212 s over nominal-time alignment. The complete system completes 21/50 simulated multi-site missions (42.0%) and is further validated on a physical articulated UAM.

发表机构

• Nanyang Technological University（南洋理工大学）
• Tongji University（同济大学）
• Shanghai Institute of Intelligent Science and Technology（上海自主智能无人系统科学中心）
• Beihang University（北京航空航天大学）
• VinUniversity

机构由 AI 辅助整理，请以论文原文为准。