arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.13445cs.LGcs.AI
收藏

任务无关的决策Transformer用于多类型智能体控制与联邦拆分训练

Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training

Tsinghua University(清华大学);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)

Zhiyuan Wang, Bokui Chen, Xiaoyang Qu, Zhenhou Hong, Jing Xiao, Jianzong Wang

AI总结:

针对个性化智能体状态与动作空间差异导致的联邦学习聚合难题,提出联邦拆分决策Transformer(FSDT),采用两阶段训练,在D4RL基准上实现高效、隐私保护的多类型智能体决策,显著降低通信与计算开销。

摘要:

随着人工智能的快速发展,开发知识丰富且个性化的智能体变得越来越普遍。然而,个性化智能体之间状态变量和动作空间的固有差异,给传统联邦学习算法带来了显著的聚合挑战。为了解决这些挑战,我们引入了联邦拆分决策Transformer(FSDT),这是一个专门为AI智能体决策任务设计的创新框架。FSDT框架通过利用分布式数据进行训练,同时保护数据隐私,擅长应对个性化智能体的复杂性。它采用两阶段训练过程,客户端智能体上设有本地嵌入和预测模型,服务器上设有全局Transformer解码器模型。我们使用基准D4RL数据集的全面评估,突显了我们的算法在个性化智能体的联邦拆分学习中的优越性能,同时与传统集中式训练方法相比,显著降低了通信和计算开销。FSDT框架展示了在自动驾驶决策系统等应用中实现高效且保护隐私的协作学习的强大潜力。我们的研究结果强调了FSDT框架在有效利用分布式离线强化学习数据以支持强大的多类型智能体决策系统方面的有效性。

英文摘要:

With the rapid advancements in artificial intelligence, the development of knowledgeable and personalized agents has become increasingly prevalent. However, the inherent variability in state variables and action spaces among personalized agents poses significant aggregation challenges for traditional federated learning algorithms. To tackle these challenges, we introduce the Federated Split Decision Transformer (FSDT), an innovative framework designed explicitly for AI agent decision tasks. The FSDT framework excels at navigating the intricacies of personalized agents by harnessing distributed data for training while preserving data privacy. It employs a two-stage training process, with local embedding and prediction models on client agents and a global transformer decoder model on the server. Our comprehensive evaluation using the benchmark D4RL dataset highlights the superior performance of our algorithm in federated split learning for personalized agents, coupled with significant reductions in communication and computational overhead compared to traditional centralized training approaches. The FSDT framework demonstrates strong potential for enabling efficient and privacy-preserving collaborative learning in applications such as autonomous driving decision systems. Our findings underscore the efficacy of the FSDT framework in effectively leveraging distributed offline reinforcement learning data to enable powerful multi-type agent decision systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑