面向分布外场景的联邦蒸馏：基于领域感知代理的方法

中文摘要 AI 辅助

联邦学习是一种分布式机器学习范式，它通过聚合本地客户端的模型来训练全局模型，且无需共享各客户端的私有数据。联邦蒸馏（Federated Distillation, FD）基于该范式，利用知识蒸馏在代理数据上交换软预测而非模型参数，从而实现更高效的通信并支持异构模型协作。然而，在分布内（In-Distribution）数据上训练的联邦蒸馏模型难以适配分布外（Out-of-Distribution, OOD）场景。本文提出一种领域感知代理选择框架，以更好地适配代理数据解决分布外问题。实验结果表明，所提模型在标准基准测试中，针对有代理数据和无代理数据的分布外分布偏移挑战，分别较现有工作取得了平均82.9%和80.6%的性能提升，相关代码与数据已在指定URL公开。

英文摘要

Federated Learning is a distributed machine learning paradigm that trains a global model by aggregating local clients without sharing private data of each client. Federated Distillation (FD) builds upon this paradigm by leveraging knowledge distillation to exchange soft predictions on proxy data instead of model parameters, enabling more efficient communication and supporting heterogeneous model collaboration. However, FD models trained on In-Distribution data are hardly adapted to Out-of-Distribution (OOD) scenarios. In this paper, we propose a domain-aware proxy selection framework to better adopt proxy data for OOD problems. The experimental results show that the proposed models effectively address the challenges of distribution shifts under OOD with and without proxy data by achieving average 82.9\% and 80.6\% over existing works on standard benchmarks. The codes and data are released in https://anonymous.4open.science/r/DPS-FD-8596.

发表机构

• School of Information Science and Engineering, Yunnan University（云南大学信息科学与工程学院）

机构由 AI 辅助整理，请以论文原文为准。