发表机构
University of Southern California（南加州大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
ViBe通过后训练框架，利用预训练视觉编码器和低秩适配器，将运动跟踪器适应为感知控制器，实现零样本模拟到真实迁移，在多项任务中展现鲁棒性。
AI 中文摘要
运动跟踪为人体全身控制提供了一种可扩展的方法。通过设计，由此产生的跟踪器缺乏外部感知反馈，因此对环境做出反应仍然是更高级规划器的责任。现有的感知控制器从头开始训练仅基于几何的编码器，以语义换取模拟到真实的简便性，并且通常依赖教师-学生蒸馏来处理感兴趣的任务。我们提出了ViBe，一个用于将运动跟踪器适应感知控制任务的后训练框架。我们利用预训练的视觉编码器，结合多查询提取模块来学习与任务相关的感知反馈。该反馈通过低秩适配器嫁接到跟踪器的输入上，实现参数高效的微调。给定任务奖励和参考数据集，该模块化控制器可以直接通过策略优化进行适应。在四项任务中，ViBe展示了零样本模拟到真实的迁移，涵盖路缘和跑酷的感知行走、重定位立方体、全物体移动操作以及躲避球，在户外、低光照和RGB干扰条件下均表现出视觉鲁棒性。最后，我们用一个刻意简单的规划器解决了目标导向的重定位立方体任务，证明了通过我们的方法适应的感知控制器的有效性。
英文摘要
Motion tracking provides a scalable recipe for humanoid whole-body control. By design, the resulting trackers lack exteroceptive feedback hence reacting to the environment remains the responsibility of a higher-level planner. Existing perceptive controllers train geometry-only encoders from scratch, trading semantics for sim-to-real ease, and typically rely on teacher-student distillation for a task of interest. We present ViBe, a post-training framework for adapting motion trackers to perceptive control tasks. We leverage pre-trained visual encoders with a multi-query extractor module to learn task-relevant perceptive feedback. This feedback is grafted onto the tracker's input via low-rank adapters, enabling parameter-efficient fine-tuning. Given a task reward and a reference dataset, this modular controller can be adapted directly via policy optimization. Across four tasks, ViBe shows zero-shot sim-to-real transfer spanning perceptive walking on curbs and parkour, Repose Cube, omni-object loco-manipulation, and dodgeball, with visually robust performance across outdoor, low-light, and RGB distractor conditions. Finally, we solve a goal-oriented Repose Cube task with a deliberately simple planner, demonstrating the efficacy of perceptive controllers, adapted by our approach.
Commentsproject page: https://lok-i.github.io/vibe-control/
