arXiv 2401.116662025-09-03cs.LGcs.AI

P2DT: 采用渐进式提示决策Transformer缓解任务增量学习中的遗忘问题

P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer

Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院）；Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China（平安科技（深圳）有限公司）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室）

Zhiyuan Wang, Xiaoyang Qu, Jing Xiao, Bokui Chen, Jianzong Wang

备注：: Accepted by the 49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大模型控制的智能体在任务增量学习中面临的灾难性遗忘问题，提出渐进式提示决策Transformer P2DT，通过动态附加决策令牌培育任务专属策略，利用全任务轨迹保留旧知识，有效缓解遗忘且适配任务规模扩展。

摘要：

灾难性遗忘对管理由大模型控制的智能体构成了重大挑战，会导致这些智能体在面对新任务时性能下降。在本研究中，我们提出了一种新的解决方案——渐进式提示决策Transformer（Progressive Prompt Decision Transformer，P2DT）。该方法通过在新任务训练期间动态附加决策令牌来增强基于Transformer的模型，从而培育针对特定任务的策略。我们的方法可缓解持续强化学习和离线强化学习场景中的遗忘问题。此外，P2DT利用通过传统强化学习从所有任务中收集的轨迹，并在训练期间生成新的特定任务令牌，从而保留从先前研究中获得的知识。初步结果表明，我们的模型有效缓解了灾难性遗忘，并且能随着任务环境的增加良好扩展。

英文摘要：

Catastrophic forgetting poses a substantial challenge for managing intelligent agents controlled by a large model, causing performance degradation when these agents face new tasks. In our work, we propose a novel solution - the Progressive Prompt Decision Transformer (P2DT). This method enhances a transformer-based model by dynamically appending decision tokens during new task training, thus fostering task-specific policies. Our approach mitigates forgetting in continual and offline reinforcement learning scenarios. Moreover, P2DT leverages trajectories collected via traditional reinforcement learning from all tasks and generates new task-specific tokens during training, thereby retaining knowledge from previous studies. Preliminary results demonstrate that our model effectively alleviates catastrophic forgetting and scales well with increasing task environments.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。