学习适应性跨主体视觉运动策略与对比提示协调
Learning Adaptive Cross-Embodiment Visuomotor Policy with Contrastive Prompt Orchestration
AI总结：
CAPO通过对比提示学习和自适应提示协调，提升跨主体视觉运动策略的适应性和样本效率。
摘要：
学习具有身体的视觉运动策略仍然是一个严峻的挑战，尤其是在面对跨主体变化时，如多样化的传感器配置和动态属性。传统学习方法往往难以将任务相关的特征与领域特定的变化（例如光照、视野和旋转）区分开来，导致样本效率低下并在未见过的环境中出现灾难性失败。为此，我们提出了ContrAstive Prompt Orchestration（CAPO），一种新的学习视觉运动策略的方法，它集成了对比提示学习和自适应提示协调。在提示学习方面，我们设计了一种混合对比学习策略，整合了视觉、时间动作和文本目标，以建立一个可学习的提示池，其中每个提示诱导一个包含细粒度领域因素的视觉表示。基于这些学习的提示，我们引入了一种自适应提示协调机制，该机制根据当前的观察动态聚合这些提示。这使代理能够通过识别主导的领域因素即时构建最优的状态表示。因此，策略优化被有效地屏蔽了无关干扰，防止了对源领域的过度拟合问题。广泛的实验表明，CAPO在样本效率和渐近性能方面显著优于最先进的基线。关键的是，它在未见过的目标领域中表现出卓越的零样本适应能力，这些领域特征剧烈（例如光照）和物理变化（例如视野和旋转），验证了其作为跨主体视觉运动策略适应的可行解决方案的有效性。
英文摘要：
Learning adaptive visuomotor policies for embodied agents remains a formidable challenge, particularly when facing cross-embodiment variations such as diverse sensor configurations and dynamic properties. Conventional learning approaches often struggle to separate task-relevant features from domain-specific variations (e.g., lighting, field-of-view, and rotation), leading to poor sample efficiency and catastrophic failure in unseen environments. To bridge this gap, we propose ContrAstive Prompt Orchestration (CAPO), a novel approach for learning visuomotor policies that integrates contrastive prompt learning and adaptive prompt orchestration. For prompt learning, we devise a hybrid contrastive learning strategy that integrates visual, temporal action, and text objectives to establish a pool of learnable prompts, where each prompt induces a visual representation encapsulating fine-grained domain factors. Based on these learned prompts, we introduce an adaptive prompt orchestration mechanism that dynamically aggregates these prompts conditioned on current observations. This enables the agent to adaptively construct optimal state representations by identifying dominant domain factors instantaneously. Consequently, the policy optimization is effectively shielded from irrelevant interference, preventing the common issue of overfitting to source domains. Extensive experiments demonstrate that CAPO significantly outperforms state-of-the-art baselines in sample efficiency and asymptotic performance. Crucially, it exhibits superior zero-shot adaptation across unseen target domains characterized by drastic environmental (e.g., illumination) and physical shifts (e.g., field-of-view and rotation), validating its effectiveness as a viable solution for cross-embodiment visuomotor policy adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
