arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.09494cs.LG
收藏

论去中心化数据下在线模型选择中协作的必要性

On the Necessity of Collaboration for Online Model Selection with Decentralized Data

Harbin Institute of Technology Shenzhen(哈尔滨工业大学深圳校区);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)

Junfan Li, Zheshun Wu, Zenglin Xu, Irwin King

AI总结:

该研究从计算约束视角探讨去中心化数据下在线模型选择的客户端协作必要性,证明遗憾值上下界,明确无计算约束时协作不必要、计算成本限$o(K)$时必要,还提出含三项新技术的联邦算法并改进遗憾界。

摘要:

我们研究了$M$个客户端上基于去中心化数据的在线模型选择问题,并探讨了客户端之间协作的必要性。已有工作提出了多种联邦算法,但未证明其必要性,我们从计算约束的全新视角回答了这一问题。我们证明了遗憾值的下界,提出了一种联邦算法并分析了其上界。我们的结果表明:(i)当客户端无计算约束时,协作是不必要的;(ii)当每个客户端的计算成本被限制在$o(K)$(其中$K$为候选假设空间的数量)时,协作是必要的。我们阐明了已有分布式在线多核学习联邦算法中协作的非必要性,并以更低的计算和通信成本改进了遗憾值界。我们的算法依赖三项新技术,包括改进的鞅伯恩斯坦不等式、联邦在线镜像下降框架,以及模型选择与预测的解耦,这些技术本身可能具有独立研究价值。

英文摘要:

We consider online model selection with decentralized data over $M$ clients, and study the necessity of collaboration among clients. Previous work proposed various federated algorithms without demonstrating their necessity,while we answer the question from a novel perspective of computational constraints. We prove lower bounds on the regret, and propose a federated algorithm and analyze the upper bound.Our results show (i) collaboration is unnecessary in the absence of computational constraints on clients; (ii) collaboration is necessary if the computational cost on each client is limited to $o(K)$, where $K$ is the number of candidate hypothesis spaces. We clarify the unnecessary nature of collaboration in previous federated algorithms for distributed online multi-kernel learning,and improve the regret bounds at a smaller computational and communication cost. Our algorithm relies on three new techniques including an improved Bernstein's inequality for martingale, a federated online mirror descent framework, and decoupling model selection and prediction, which might be of independent interest.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑