arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.04099cs.LG
收藏

多目标多解传输(Many-Objective Multi-Solution Transport)

Many-Objective Multi-Solution Transport

University of Maryland(马里兰大学);University of Chicago(芝加哥大学);Carnegie Mellon University(卡内基梅隆大学);University of Washington, Seattle(华盛顿大学西雅图分校)

Ziyue Li, Tian Li, Virginia Smith, Jeff Bilmes, Tianyi Zhou

AI总结:

针对多目标优化中目标数多于解数时性能差、目标易被忽略的问题,提出MosT框架,通过双层优化与最优传输定义权重,生成覆盖帕累托前沿的多样化解,在多类任务中性能优于基线。

摘要:

在机器学习中,联合优化多个目标(由任务或客户端实例化)与少量帕累托平稳解(模型)的性能至关重要。然而,以往的多目标优化方法通常仅关注少量目标,无法扩展到目标数量多于解数量的多目标场景,导致性能不佳或部分目标被忽略。我们提出了多目标多解传输框架(Many-objective multi-solution Transport, MosT),该框架可在多目标的帕累托前沿中找到多个多样化的解。我们的核心思路是寻求多个解,每个解作为领域专家聚焦于特定的目标子集,同时所有解共同覆盖全部目标。MosT将该问题建模为每个解的加权目标双层优化问题,其中权重由目标与解之间的最优传输定义。我们的算法确保能够收敛到互补目标子集的帕累托平稳解。在联邦学习、多任务学习以及大语言模型(LLM)的提示混合学习等一系列应用中,MosT显著优于强基线方法,提供了高质量、多样化的解,能够刻画整个帕累托前沿,从而在多个目标间实现平衡的权衡。

英文摘要:

Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning. However, previous multi-objective optimization methods often focus on a few number of objectives and cannot scale to many objectives that outnumber the solutions, leading to either subpar performance or ignored objectives. We introduce Many-objective multi-solution Transport (MosT), a framework that finds multiple diverse solutions in the Pareto front of many objectives. Our insight is to seek multiple solutions, each performing as a domain expert and focusing on a specific subset of objectives while collectively covering all of them. MosT formulates the problem as a bi-level optimization of weighted objectives for each solution, where the weights are defined by an optimal transport between the objectives and solutions. Our algorithm ensures convergence to Pareto stationary solutions for complementary subsets of objectives. On a range of applications in federated learning, multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly outperforms strong baselines, delivering high-quality, diverse solutions that profile the entire Pareto frontier, thus ensuring balanced trade-offs across many objectives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑