ClothPPO：一种用于机器人布料操作、具备观测对齐动作空间的近端策略优化增强框架
ClothPPO: A Proximal Policy Optimization Enhancing Framework for Robotic Cloth Manipulation with Observation-Aligned Action Spaces
AI总结：
本文提出 ClothPPO 框架，将布料操作建模为 POMDP，并通过监督预训练与 PPO 在观测对齐的百万级动作空间中增强策略，提升机器人布料展开性能。
摘要：
基于视觉的机器人布料展开近年来取得了巨大进展。然而，以往工作主要依赖价值学习，尚未充分探索基于策略的技术。最近，强化学习在大型语言模型上的成功表明，策略梯度算法能够在巨大动作空间中增强策略。本文提出 ClothPPO，这是一个采用基于 actor-critic 架构的策略梯度算法的框架，用于在展开衣物任务中，借助与观测对齐、规模达 10^6 的巨大动作空间来增强预训练模型。为此，我们将布料操作问题重新定义为部分可观测马尔可夫决策过程。框架采用监督预训练阶段来训练策略的基线模型；在第二阶段，利用近端策略优化（PPO）在观测对齐的动作空间内引导该监督模型。通过优化和更新策略，所提方法在软体操作任务下增加了布料展开时的衣物表面积。实验结果表明，所提出的框架能够进一步提升其他最先进方法的展开性能。
英文摘要：
Vision-based robotic cloth unfolding has made great progress recently. However, prior works predominantly rely on value learning and have not fully explored policy-based techniques. Recently, the success of reinforcement learning on the large language model has shown that the policy gradient algorithm can enhance policy with huge action space. In this paper, we introduce ClothPPO, a framework that employs a policy gradient algorithm based on actor-critic architecture to enhance a pre-trained model with huge 10^6 action spaces aligned with observation in the task of unfolding clothes. To this end, we redefine the cloth manipulation problem as a partially observable Markov decision process. A supervised pre-training stage is employed to train a baseline model of our policy. In the second stage, the Proximal Policy Optimization (PPO) is utilized to guide the supervised model within the observation-aligned action space. By optimizing and updating the strategy, our proposed method increases the garment's surface area for cloth unfolding under the soft-body manipulation task. Experimental results show that our proposed framework can further improve the unfolding performance of other state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
