arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.16828cs.LGcs.CL
收藏

共享表示的贝尔边界

The Bell Boundary of Shared Representations

Tianjin University(天津大学)

Yang Wu, Hanyu Zhao, Yazhou Zhang, Yuexian Hou

AI总结:

本文从贝尔定理视角研究多任务学习中共享表示的容量边界,引入NCnet模型发现容量依赖的贝尔边界转变,该现象在CNNs、PLMs、LLMs中均存在,可用于诊断多任务模型的相关问题。

摘要:

当共享表示不足以同时支持多个任务时,共享表示会在何种情况下失效?这一问题是多任务学习模型的核心。尽管共享表示中的任务干扰已被广泛观察到,但其发生时机、可测量形式及容量边界仍知之甚少。本文中,我们从贝尔定理的视角重新审视该问题。我们证明,若四个二元任务结果可由同一共享表示联合表示,则它们的相关性必须满足经典CHSH边界|S|≤2。我们引入NCnet,一种用于研究表示容量如何影响任务兼容性的受控多任务模型。实验揭示了清晰的容量依赖转变:一旦出现非平凡的任务拟合,受限容量可产生|S|>2;随着容量进一步增加,这些超出值逐渐消失,S返回有界区域。这一系统模式提供了证据,表明有限的表示容量是观察到的不兼容性的重要来源。我们在CNNs、PLMs和LLMs中观察到类似转变,表明该现象超出了受控NCnet的设置范围。我们将这种容量依赖转变称为贝尔边界,为多任务模型中的表示瓶颈、任务不兼容性及容量分配提供了可测量的诊断依据。

英文摘要:

When does a shared representation become insufficient for jointly supporting multiple tasks? This question lies at the heart of multi-task learning models. Although task interference in shared representations has been widely observed, its onset, measurable form, and capacity boundary remain poorly understood. In this paper, we revisit this problem through the lens of Bell's theorem. We show that if four binary task outcomes can be jointly represented by a common shared representation, their correlations must satisfy the classical CHSH bound $|S|\leq2$. We introduce NCnet, a controlled multi-task model for studying how representational capacity affects task compatibility. Experiments reveal a clear capacity-dependent transition. Once nontrivial task fitting emerges, constrained capacity can yield $|S|>2$. As capacity further increases, these exceedances progressively disappear and $S$ returns to the bounded region. This systematic pattern provides evidence that limited representational capacity is an important source of the observed incompatibility. We observe similar transitions across CNNs, PLMs, and LLMs, suggesting that the phenomenon extends beyond the controlled NCnet setting. We refer to this capacity-dependent transition as the Bell Boundary, providing a measurable diagnostic for representation bottlenecks, task incompatibility, and capacity allocation in multi-task models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑