发表机构
Harvard Medical School（哈佛医学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究多任务学习中负迁移与容量、冗余的关系，通过容量 - 冗余恒等式及相关结果，如聚类差距分解和梯度 - TC 桥梁，证明聚类 LoRA 可降低残余耦合，优于随机划分，有显著收益。
AI 中文摘要
在多任务学习（MTL）中，负迁移通常被视为一种优化假象，但它也可被视为共享容量有限和任务冗余薄弱的结果。我们通过容量 - 冗余（CR）恒等式来研究这种效应，该恒等式将每个任务的预测信息之和分解为联合预测信息（包括通过全相关（TC）定义的标签冗余）和一个残余耦合项（量化共享表示未解决的干扰）。此外，我们展示了两个关键结果：（i）聚类差距分解，给出聚类共享优于全局共享的充要条件；（ii）高斯多任务模型中的梯度 - TC 桥梁，从形式上证明梯度余弦相似度可作为冗余排序的代理。从经验上看，我们从验证残余相关性估计残余耦合$\Delta$，表明聚类 LoRA 显著降低$\widehat{\Delta}$，优于大小匹配的随机划分，并在多种子置信区间下带来统计上显著的收益。
英文摘要
In multi-task learning (MTL) negative transfer is often considered as an optimization artifact, but it can also be viewed as a consequence of limited shared capacity and weak task redundancy. We investigate this effect through a Capacity--Redundancy (CR) identity that decomposes the sum of per-task predictive informations into joint predictive information that includes label redundancy defined via total correlation (TC), and a residual coupling term that quantifies interference left unresolved by the shared representation. Additionally, we show two key results: (i) a clustering-gap decomposition that gives a necessary and sufficient condition for clustered sharing to outperform global sharing, and (ii) a gradient--TC bridge in a Gaussian multi-task model that formally justifies gradient cosine similarity as a proxy for redundancy ordering. Empirically, we estimate the residual coupling $Δ$ from validation residual correlations, showing that clustered LoRA substantially reduces $\widehatΔ$, outperforms size-matched random partitions, and results in statistically significant gains with multi-seed confidence intervals.
CommentsAccepted in 42nd Conference on Uncertainty in Artificial Intelligence (UAI) 2026
