arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.00299cs.LGcs.SYeess.IVeess.SY
收藏

通过最大编码率降低的联邦表示学习

Federated Representation Learning via Maximal Coding Rate Reduction

University of Pennsylvania(宾夕法尼亚大学)

Juan Cervino, Navid NaderiAlizadeh, Alejandro Ribeiro

摘要:

我们提出了一种联邦方法来学习低维表示,从分布在几个客户端的数据集。特别是,我们远离了联邦学习中常用的交叉熵损失,并试图通过最大编码率降低(MCR2)的原则以分散的方式学习数据的共享低维表示。我们提出的方法,我们称之为FLOW,利用MCR2作为选择的目标,因此产生了类间判别和类内可压缩的表示。我们从理论上表明,我们的分布式算法实现了一阶稳定点。此外,我们证明,通过数值实验,实用的学习低维表示。

英文摘要:

We propose a federated methodology to learn low-dimensional representations from a dataset that is distributed among several clients. In particular, we move away from the commonly-used cross-entropy loss in federated learning, and seek to learn shared low-dimensional representations of the data in a decentralized manner via the principle of maximal coding rate reduction (MCR2). Our proposed method, which we refer to as FLOW, utilizes MCR2 as the objective of choice, hence resulting in representations that are both between-class discriminative and within-class compressible. We theoretically show that our distributed algorithm achieves a first-order stationary point. Moreover, we demonstrate, via numerical experiments, the utility of the learned low-dimensional representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑