PACT: 具身操作中扩散策略的自我演化物理安全对齐
PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
AI总结：
提出PACT框架，通过自演化后训练将预训练扩散策略投影到约束可行区域，无需演示数据或任务奖励，在降低31.0%安全违规的同时提升30.7%任务成功率。
摘要：
扩散策略在机器人操作中取得了显著成功，但常常无法满足安全部署所需的严格物理约束。现有方法要么在训练期间过早施加安全约束，要么在测试时通过外部护栏被动应对，限制了策略的表达能力和整体可扩展性。我们提出物理安全对齐约束轨迹（PACT），这是一个自我演化的后训练框架，将预训练扩散策略投影到约束可行区域，无需访问演示数据或任务奖励。PACT通过跨时间步密集监督的反向KL目标将约束梯度蒸馏到扩散模型中。它采用课程学习逐步收紧约束，同时保持理论上界定的策略偏移和单调改进，减轻了灾难性遗忘带来的安全-性能权衡。在模拟和真实世界的具身操作基准测试中，PACT平均减少31.0%的安全违规，同时将任务成功率提升30.7%。
英文摘要：
Diffusion policies have achieved remarkable success in robotic manipulation, yet they often fail to satisfy strict physical constraints required for safe deployment. Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time, limiting policy expressivity and overall scalability. We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions without accessing demonstration data or task rewards. PACT distills constraint gradients into the diffusion model through a reverse-KL objective with dense supervision across timesteps. It incorporates a curriculum that progressively tightens constraints while maintaining theoretically bounded policy shift and monotone improvement, mitigating the safety-performance trade-off from catastrophic forgetting. On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
