NeurIPS

$\boldsymbolλ$-Orthogonality Regularization for Compatible Representation Learning

λ-正交性正则化用于兼容表示学习

Simone Ricci, Niccolò Biondi, Federico Pernici, Ioannis Patras, Alberto Del Bimbo

机构 * DINFO (Department of Information Engineering), University of Florence, Italy（意大利佛罗伦萨大学信息工程系） ； MICC (Media Integration and Communication Center)（媒体整合与通信中心） ； Queen Mary University of London, UK（英国伦敦女王学院） 纠错

AI总结 本文提出λ-正交性正则化方法，通过学习仿射变换在保持原有表示的同时实现分布特定的适应，验证了其在不同架构和数据集上的有效性，保持了零样本性能并确保模型更新的兼容性。

Comments Accepted at NeurIPS2025

Journal ref Advances in Neural Information Processing Systems 38 (NeurIPS 2025), pp. 29036-29063

Distributional Autoencoders Know the Score

分布式自编码器知晓得分

Andrej Leban

机构 * Department of Statistics, University of Michigan（密歇根大学统计学系） 纠错

AI总结 本文提出分布式主元自编码器，通过理论保证实现分布正确重建与编码可解释性，证明模型能同时学习数据分布和内在维度。

Comments NeurIPS 2025 - camera-ready version

Journal ref Advances in Neural Information Processing Systems 38 (NeurIPS 2025), 2025

Bayes optimal learning of attention-indexed models

贝叶斯最优学习注意力索引模型

Fabrizio Boncoraglio, Emanuele Troiani, Vittorio Erba, Lenka Zdeborová

机构 * Statistical Physics of Computation Laboratory, École polytechnique fédérale de Lausanne (EPFL)（计算物理学实验室，瑞士联邦理工学院（EPFL）） 纠错

AI总结 本文提出注意力索引模型，通过理论分析和算法设计，探讨深度注意力层的贝叶斯最优学习问题。

Journal ref NeurIPS 2025

The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks

核路：在过参数化二次网络中ERM的尖锐渐近性

Vittorio Erba, Emanuele Troiani, Lenka Zdeborová, Florent Krzakala

机构 * Statistical Physics of Computation Laboratory（计算物理学统计力学实验室） ； Information, Learning and Physics Laboratory（信息、学习与物理实验室） 纠错

AI总结 该研究通过将过参数化二次网络的ERM问题转化为凸矩阵感知任务，揭示了低秩结构对容量控制的影响，并确定了目标函数宽度对可学习性的作用。

Journal ref NeurIPS 2025

Stitch and Tell: A Structured Multimodal Data Augmentation Method for Spatial Understanding

拼接与讲述：一种结构化多模态数据增强方法用于空间理解

Hang Yin, Xiaomin He, PeiWen Yuan, Yiwei Li, Jiayi Shi, Wenxiao Fan, Shaoxiong Feng, Kan Li

机构 * School of Computer Science, Beijing Institute of Technology（北京理工大学计算机科学学院） ； School of Software and Microelectronics, Peking University（北京大学软件与微电子学院） ； Xiaohongshu Inc（小红书公司） 纠错

AI总结 Stitch and Tell通过结构化空间监督提升视觉-语言模型的空间理解能力，有效缓解空间幻觉并提高相关任务性能。

Journal ref Advances in Neural Information Processing Systems 38 (NeurIPS 2025)

The Computational Advantage of Depth: Learning High-Dimensional Hierarchical Functions with Gradient Descent

Yatin Dandi, Luca Pesce, Lenka Zdeborová, Florent Krzakala

机构 * Information, Learning and Physics Laboratory. Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland.（信息、学习与物理实验室。瑞士洛桑联邦理工学院（EPFL）） ； Statistical Physics of Computation Laboratory. Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland.（计算统计物理实验室。瑞士洛桑联邦理工学院（EPFL）） 纠错

Journal ref NeurIPS 2025 (Spotlight)

Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data

等变稀疏自编码器：对称数据上神经网络的机制可解释性

Ege Erdogan, Ana Lucic

机构 * University of Amsterdam（阿姆斯特丹大学） 纠错

AI总结 该研究针对稀疏自编码器在对称数据上的不可识别问题，提出等变稀疏自编码器，可避免缺陷并发现更有用的下游任务特征，表明重构质量与特征实用性在对称下可能负相关。

Comments NeurIPS 2025 Mechanistic Interpretability and UniReps workshops

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks

Luca Arnaboldi, Bruno Loureiro, Ludovic Stephan, Florent Krzakala, Lenka Zdeborova

Journal ref NeurIPS 2025

Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions

Yizhou Xu, Florent Krzakala, Lenka Zdeborová

机构 * Statistical Physics of Computation Laboratory (SPOC), EPFL, Switzerland（计算统计物理实验室（SPOC），瑞士联邦理工学院） ； Information, Learning, and Physics Laboratory (IDEPHICS), EPFL, Switzerland（信息、学习与物理实验室（IDEPHICS），瑞士联邦理工学院） 纠错

Journal ref NeurIPS 2025

Multi-layer State Evolution Under Random Convolutional Design

Mara Daniels, Cédric Gerbelot, Florent Krzakala, Lenka Zdeborová

Comments Accepted to NeurIPS 2022

Journal ref Advances in Neural Information Processing Systems (2022), vol 52, pages 7089--7102

Bayes-optimal learning of an extensive-width neural network from quadratically many samples

Antoine Maillard, Emanuele Troiani, Simon Martin, Florent Krzakala, Lenka Zdeborová

Comments 47 pages

Journal ref Advances in Neural Information Processing Systems 37 (NeurIPS 2024)

A phase transition between positional and semantic learning in a solvable model of dot-product attention

Hugo Cui, Freya Behrens, Florent Krzakala, Lenka Zdeborová

Journal ref Advances in Neural Information Processing Systems 37 (NeurIPS 2024)

High-dimensional Asymptotics of Denoising Autoencoders

Hugo Cui, Lenka Zdeborová

Journal ref Advances in Neural Information Processing Systems 36 (2023)

Universality laws for Gaussian mixtures in generalized linear models

Yatin Dandi, Ludovic Stephan, Florent Krzakala, Bruno Loureiro, Lenka Zdeborová

Journal ref Advances in Neural Information Processing Systems 36 (2023)

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

Comments 20 pages

Journal ref Advances in Neural Information Processing Systems (2022), vol 35, pages {23244--23255)

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Luca Pesce, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová

Comments NeurIPS camera-ready version

Journal ref Advances in Neural Information Processing Systems (2022), vol 35, pages 27087--27099