arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.13871cs.LG
收藏

通过密度估计改进异构模型复用

Improving Heterogeneous Model Reuse by Density Estimation

Wuhan University(武汉大学);Hubei Luojia Laboratory(湖北珞珈实验室);Beijing Institute of Technology(北京理工大学);JD Explore Academy, JD.com, Inc.(京东集团京东探索研究院);Washington University in St. Louis(圣路易斯华盛顿大学);The University of Sydney(悉尼大学)

Anke Tang, Yong Luo, Han Hu, Fengxiang He, Kehua Su, Bo Du, Yixin Chen, Dacheng Tao

AI总结:

本文研究多方学习中的异构模型复用问题,提出通过估计本地数据密度设计辅助模型,并引入多方交叉熵损失校准未充分预训练的模型,实验证明该方法在合成和基准数据上具有优越性。

摘要:

本文研究多方学习,旨在利用不同参与方的私有数据来学习一个模型。模型复用是多方学习的一种有前景的解决方案,它假设每个参与方都已经训练了一个本地模型。考虑到不同参与方之间潜在的样本选择偏差,一些异构模型复用方法已被开发出来。然而,尽管这些方法利用了预训练的本地分类器,但本地数据的特征并未得到充分利用。这促使我们估计本地数据的密度,并设计一个辅助模型与本地分类器一起进行复用。为了应对某些本地模型未被充分预训练的场景,我们进一步设计了一种多方交叉熵损失用于校准。在现有工作的基础上,我们从决策理论的角度解决了异构模型复用的一个挑战性问题,并利用了密度估计的最新进展。在合成数据和基准数据上的实验结果证明了所提方法的优越性。

英文摘要:

This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assuming that a local model has been trained for each party. Considering the potential sample selection bias among different parties, some heterogeneous model reuse approaches have been developed. However, although pre-trained local classifiers are utilized in these approaches, the characteristics of the local data are not well exploited. This motivates us to estimate the density of local data and design an auxiliary model together with the local classifiers for reuse. To address the scenarios where some local models are not well pre-trained, we further design a multiparty cross-entropy loss for calibration. Upon existing works, we address a challenging problem of heterogeneous model reuse from a decision theory perspective and take advantage of recent advances in density estimation. Experimental results on both synthetic and benchmark data demonstrate the superiority of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑