首页/论文/2012.04839

arXiv 2012.048392020-12-09cs.LG

Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation

University of Edinburgh（爱丁堡大学）

Chenyang Zhao, Timothy Hospedales

英文摘要：

In reinforcement learning, domain randomisation is an increasingly popular technique for learning more general policies that are robust to domain-shifts at deployment. However, naively aggregating information from randomised domains may lead to high variance in gradient estimation and unstable learning process. To address this issue, we present a peer-to-peer online distillation strategy for RL termed P2PDRL, where multiple workers are each assigned to a different environment, and exchange knowledge through mutual regularisation based on Kullback-Leibler divergence. Our experiments on continuous control tasks show that P2PDRL enables robust learning across a wider randomisation distribution than baselines, and more robust generalisation to new environments at testing.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Dual Policy Distillation

Texas A&M University（德州农工大学）

Policy distillation, which transfers a teacher policy to a student policy has achieved great success in challenging tasks of deep reinforcement learning. This teacher-student framework requires a well-trained teacher model which is computationally expensive. Moreover, the performance of the student model could be limited by the teacher model if the teacher model is not optimal. In the light of collaborative learning, we study the feasibility of involving joint intellectual efforts from diverse perspectives of student models. In this work, we introduce dual policy distillation(DPD), a student-s…

2020-06-07cs.AIcs.LG

在分歧处定位信用：路径条件自蒸馏用于LLM推理

Localizing Credit at the Divergence: Path-Conditioned Self-Distillation for LLM Reasoning

Department of Electrical and Computer Engineering, George Washington University（乔治华盛顿大学电气与计算机工程系）

提出Hindsight Self-Distillation (HSD)方法，通过将教师模型条件于当前训练组中的成功同伴轨迹，在失败与成功轨迹的分歧处提供密集信用信号，提升LLM在数学和代码推理任务上的性能。

2026-06-16cs.AIcs.LG

学习协作：一种协同-去中心化框架用于点对点大语言模型联邦

Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation

KNEXA-FL通过协同去中心化框架提升点对点大语言模型联邦学习的效率与稳定性

2026-01-23cs.AIcs.CRcs.DC

Online Knowledge Distillation with Diverse Peers

Zhejiang University（浙江大学）；ZJU-LianlianPay Joint Research Center（浙江大学-连连支付联合研究中心）；Zhejiang University of Technology（浙江工业大学）

Distillation is an effective knowledge-transfer technique that uses predicted distributions of a powerful teacher model as soft targets to train a less-parameterized student model. A pre-trained high capacity teacher, however, is not always available. Recently proposed online variants use the aggregated intermediate predictions of multiple student models as targets to train each student model. Although group-derived targets give a good recipe for teacher-free distillation, group members are homogenized quickly with simple aggregation functions, leading to early saturated solutions. In this wor…

2019-12-05cs.LGstat.ML

PeerAiD：从专用对等教师改进对抗蒸馏

PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor

Seoul National University（首尔大学）

针对固定教师网络对迁移对抗样本鲁棒性下降的问题，提出PeerAiD对抗蒸馏方法，同时训练专用对等网络与学生网络，在TinyImageNet上显著提升了学生网络的鲁棒性与自然准确率。

2024-05-17cs.CVcs.LG

Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach

Huazhong University of Science and Technology（华中科技大学）；Zhejiang University（浙江大学）；Peking University（北京大学）；University of Arkansas, Fayetteville（阿肯色大学费耶特维尔分校）

Peer-to-peer knowledge transfer in distributed environments has emerged as a promising method since it could accelerate learning and improve team-wide performance without relying on pre-trained teachers in deep reinforcement learning. However, for traditional peer-to-peer methods such as action advising, they have encountered difficulties in how to efficiently expressed knowledge and advice. As a result, we propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. But it is still challenging to transfer Q-function directly si…

2020-02-06cs.AIcs.LG

更多相关论文收起更多论文

Collaborative Deep Reinforcement Learning

Michigan State University（密歇根州立大学）；Rutgers University（罗格斯大学）

Besides independent learning, human learning process is highly improved by summarizing what has been learned, communicating it with peers, and subsequently fusing knowledge from different sources to assist the current learning goal. This collaborative learning procedure ensures that the knowledge is shared, continuously refined, and concluded from different perspectives to construct a more profound understanding. The idea of knowledge transfer has led to many advances in machine learning and data mining, but significant challenges remain, especially when it comes to reinforcement learning, het…

2017-02-19cs.LG

做我的导师：通过同伴反馈实现互惠LLM改进的在策略共蒸馏

Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback

KAIST（韩国科学技术院）

提出在策略共蒸馏（OPCoD）方法，通过认知门控和反馈锚定实现两个不同领域强模型间的相互教学，达到帕累托改进。

2026-06-15cs.CLcs.LG

基于同伴成功与失败的多轨迹在线策略蒸馏

Multi-Rollout On-Policy Distillation via Peer Successes and Failures

Carnegie Mellon University（卡内基梅隆大学）；Microsoft（微软）；Purdue University（普渡大学）

提出多轨迹在线策略蒸馏（MOPD），利用学生模型的本地轨迹组构造更丰富的教师信号，通过同伴成功与失败条件化提升蒸馏效果。

2026-06-01cs.AIcs.LG

Teaching on a Budget in Multi-Agent Deep Reinforcement Learning

Queen Mary University of London（伦敦玛丽女王大学）

Deep Reinforcement Learning (RL) algorithms can solve complex sequential decision tasks successfully. However, they have a major drawback of having poor sample efficiency which can often be tackled by knowledge reuse. In Multi-Agent Reinforcement Learning (MARL) this drawback becomes worse, but at the same time, a new set of opportunities to leverage knowledge are also presented through agent interactions. One promising approach among these is peer-to-peer action advising through a teacher-student framework. Despite being introduced for single-agent RL originally, recent studies show that it c…

2019-05-28cs.LGcs.MA

通过教育节点社区进行协作知识蒸馏

Collaborative Knowledge Distillation via a Learning-by-Education Node Community

Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）

提出LENC框架，通过动态角色分配和流驱动蒸馏，实现分布式DNN节点间的持续协作学习，提升分类准确率并缓解灾难性遗忘。

2026-07-01cs.LG

随机初始化网络可通过同伴到同伴共识学习

Randomly Initialized Networks Can Learn from Peer-to-Peer Consensus

研究探讨了自蒸馏在学习动态中的作用，通过训练随机初始化网络发现其在下游任务中能获得非平凡改进，分析了超参数影响及模型学习内容。

2026-04-20cs.AIcs.LG

超越最优教师：扩展与压缩推理解流形

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Moore Threads AI（摩尔线程人工智能）

该研究提出“先扩展后压缩”框架，结合RGRPO与TU-OPD等技术，构建互补教师并集并压缩，使Qwen3-1.7B学生模型在数学推理等三领域优于最强单教师，实现性能提升。

2026-07-31cs.LG

TWIST：面向高效Sim-to-Real迁移的师生世界模型蒸馏

TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfer

Applied AI Lab, Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所应用人工智能实验室）

针对基于视觉的模型强化学习的Sim-to-Real迁移中标准域随机化计算成本高的问题，提出TWIST师生世界模型蒸馏方法，利用仿真器状态信息训练教师模型并监督以域随机化图像为输入的学生模型，实现高效迁移，性能优于基线方法。

2023-11-07cs.AIcs.CVcs.LG

异步P2P闲聊学习网络中知识蒸馏的收敛理论

Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network

University of California, Santa Cruz（加州大学圣克鲁兹分校）；China University of Petroleum（中国石油大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；City University of Hong Kong（香港城市大学）；University of Virginia（弗吉尼亚大学）

本文针对异步P2P闲聊学习网络，提出了完全去中心化KD的收敛理论，证明其可将函数分歧收缩40-61倍，为异构设备的无服务器学习提供了理论支撑。

2026-09-03cs.AIcs.LG