arXiv 2509.258222025-11-11cs.RO

行动以感知，感知以行动：用于自适应策略的扩散驱动感知-行动交互

Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies

University of Alberta（阿尔伯塔大学）；Huazhong University of Science and Technology（华中科技大学）

Jing Wang, Weiting Peng, Jing Tang, Zeyu Gong, Xihua Wang, Bo Tao, Li Cheng

备注：: 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有模仿学习解耦感知与行动的问题，提出行动引导扩散策略（DP-AG），通过概率潜动力学建模感知-行动动态交互，引入行动引导SDE和循环一致对比损失，理论推导变分下界，实证显著优于现有方法，助力弥合生物适应性与人工策略学习差距。

摘要：

现有模仿学习方法将感知与行动解耦，忽略了人类自然用于自适应行为的感官表征与行动执行之间的因果互惠性。为弥合这一差距，我们提出行动引导扩散策略（DP-AG），这是一种统一的表征学习方法，通过概率潜动力学明确建模感知与行动之间的动态交互。DP-AG通过变分推理将潜观测编码为高斯后验，并使用行动引导的SDE演化它们，其中扩散策略噪声预测的向量雅可比乘积（VJP）作为结构化随机力驱动潜更新。为促进感知与行动之间的双向学习，我们引入循环一致对比损失，将噪声预测器的梯度流组织成连贯的感知-行动循环，在潜更新和行动优化中强制互洽过渡。理论上，我们推导了行动引导SDE的变分下界，并证明对比目标增强了潜轨迹和行动轨迹的连续性。实证上，DP-AG在模拟基准和真实世界UR5操作任务中显著优于现有最先进方法。因此，我们的DP-AG为弥合生物适应性与人工策略学习之间的差距提供了有前景的一步。

英文摘要：

Existing imitation learning methods decouple perception and action, which overlooks the causal reciprocity between sensory representations and action execution that humans naturally leverage for adaptive behaviors. To bridge this gap, we introduce Action-Guided Diffusion Policy (DP-AG), a unified representation learning that explicitly models a dynamic interplay between perception and action through probabilistic latent dynamics. DP-AG encodes latent observations into a Gaussian posterior via variational inference and evolves them using an action-guided SDE, where the Vector-Jacobian Product (VJP) of the diffusion policy's noise predictions serves as a structured stochastic force driving latent updates. To promote bidirectional learning between perception and action, we introduce a cycle-consistent contrastive loss that organizes the gradient flow of the noise predictor into a coherent perception-action loop, enforcing mutually consistent transitions in both latent updates and action refinements. Theoretically, we derive a variational lower bound for the action-guided SDE, and prove that the contrastive objective enhances continuity in both latent and action trajectories. Empirically, DP-AG significantly outperforms state-of-the-art methods across simulation benchmarks and real-world UR5 manipulation tasks. As a result, our DP-AG offers a promising step toward bridging biological adaptability and artificial policy learning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。