arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.09160cs.LG
收藏

面向目标的特权基通过全前缀马特罗什卡学习

Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learning

Computer Science, Columbia University(哥伦比亚大学计算机科学系)

Arghamitra Talukder, Philippe Chlenski, Itsik Pe'er

AI总结:

本文研究马特罗什卡表示学习如何诱导与任务对齐的特权基,证明全前缀MRL能高效恢复有序主方向,且坐标大小反映信息量。

摘要:

学得的表示通常对旋转变换不变,导致个体维度非可识别且可交换。我们研究马特罗什卡表示学习(MRL)如何诱导与任务对齐的特权基,区别于基于方差或正则化器的顺序。在线性设置中,我们证明全前缀MRL能恢复有序主方向,并可通过共享统计量高效计算。实证表明,MRL能产生与任务信号一致的每维结构,其中坐标大小反映信息量。

英文摘要:

Learned representations are often invariant to rotational transformations, leaving individual dimensions non-identifiable and interchangeable. We study how Matryoshka Representation Learning (MRL) induces a task-aligned privileged basis distinct from variance-based or regularizer-induced orderings. In the linear setting, we prove that full-prefix MRL recovers the ordered principal directions, and can be computed efficiently using shared statistics. Empirically, we demonstrate that MRL yields consistent per-dimension structure aligned with task signal, where coordinate magnitude reflects informativeness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑