发表机构
Tel Aviv University（特拉维夫大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对智能体数据相关导致的分布式学习性能下降问题，提出cordial学习，通过仅共享低维输出并训练本地模型提取同伴信号，在非凸目标下以概率一收敛到全局最优，实验验证了其高效性。
AI 中文摘要
我们考虑一个智能体拥有相关数据的分布式学习任务。具体而言，对于同一样本，一个智能体的标签依赖于其他智能体的输入，并且这些输入也是相关的。当智能体共享同一环境时，相关数据是现实情况。现有的去中心化方法，如联邦学习，忽略了问题的结构，在相关数据上表现不佳。另一方面，由于隐私和通信限制，集中式方法不可行。我们引入了cordial（相关且分布式）学习来解决这一差距，该方法仅在智能体之间共享低维输出，同时训练本地模型以从同伴中提取信息信号。这种分布式学习引发了一个博弈，其中每个智能体的损失函数依赖于其他智能体的模型。假设线性模型，我们证明尽管全局目标非凸，cordial学习仍以概率一收敛到全局最优解。在结构化多数字MNIST任务上的实验表明，即使在高度非线性的设置中，cordial学习仍然非常有效。
英文摘要
We consider a distributed learning task with agents that have correlated data. Specifically, the label of an agent depends on the input of other agents for the same sample, and these inputs are also correlated. Correlated data is the reality when agents share the same environment. Existing decentralized methods, such as federated learning, ignore the structure of the problem and perform poorly on correlated data. On the other hand, centralized approaches are infeasible due to privacy and communication constraints. We introduce cordial (correlated and distributed) learning to address this gap by sharing only low-dimensional outputs between the agents while training local models to extract informative signals from peers. This distributed learning induces a game in which the loss function of each agent depends on the models of others. Assuming a linear model, we prove that cordial learning converges with probability one to a globally optimal solution, despite the nonconvex global objective. Experiments on structured multi-digit MNIST tasks demonstrate that cordial learning remains highly effective even in highly nonlinear settings.
