基础模型适配器的谱秩验证
Spectral Rank Certification for Foundation Model Adapters
- Université Paris 1（巴黎第一大学）
- Faculty of Science and Technology of Tangier（丹吉尔科学与技术学院）
- Abdelmalek Essaadi University（阿卜杜勒-马利克·埃萨迪大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究针对基础模型适配器开发有限样本谱秩验证框架，提出PEFT LoRA适配器的经验空工作流程，经26个公开适配器审计发现校准有效秩远小于名义秩，且能量保留与统计惊喜对应不同问题。
AI中文摘要：
名义LoRA秩是一个设计参数，而校准的谱证据是一个独立的推断量。本文开发了一个有限样本框架，用于推断公开基础模型适配器中的有效秩结构。理论核心是固定维高斯秩一参考实验的精确卡方散度，在旋转不变参考先验下积分未知信号方向。所得级数在具体层尺寸下产生可计算的有限样本Le Cam界、数值截断的显式余项界，以及矩形Baik-Ben Arous-Peche（BBP）极限。紧流形拉普拉斯展开表明，有限样本似然证据还通过因子$s_1^{|m-n|}\times\bigcup_{i\neq2}(s_1^2-s_i^2)$依赖于主导谱间隙，这为聚类奇异值的联合校准提供了动机。基于这些结果，我们为PEFT LoRA适配器引入了经验空工作流程：因子重构、蒙特卡洛p值、阶段式与分块测试，以及模块级与语料库级BH报告。在对26个公开适配器、684个模块、6个架构家族及31770个公开检查点谱行的审计中，校准有效秩通常远小于名义秩，且与95%能量保留存在系统性差异。对n=24个样本的RoBERTa-RTE切片的测量说明了从校准秩到任务评估的测量路径，且未将该切片视为效用研究。主要实证发现是，校准有效秩通常远低于名义秩，且能量保留与统计惊喜回答不同的问题。
英文摘要：
Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity. This article develops a finite-sample framework for inferring effective rank structure in public foundation-model adapters. The theoretical core is an exact chi-square divergence for the fixed-dimensional Gaussian rank-one reference experiment, with an unknown signal direction integrated under a rotation-invariant reference prior. The resulting series yields a computable finite-sample Le Cam bound at concrete layer sizes, an explicit remainder bound for numerical truncation, and the rectangular Baik-Ben Arous-Peche (BBP) limit. A compact-manifold Laplace expansion shows that finite-sample likelihood evidence also depends on leading spectral gaps through the factor $s_1^{|m-n|}\prod_{i\ge2}(s_1^2-s_i^2)$, motivating joint calibration of clustered singular values. Building on these results, we introduce an empirical-null workflow for PEFT LoRA adapters: factor reconstruction, Monte Carlo $p$-values, stagewise and block testing, and module-wise and corpus-level BH reporting. In an audit of 26 public adapters, 684 modules, six architecture families, and 31,770 public-checkpoint spectra rows, calibrated effective rank is typically much smaller than nominal rank and differs systematically from 95\% energy retention. A measured RoBERTa-RTE slice on $n=24$ examples illustrates the measurement path from calibrated ranks to task evaluation, without treating the slice as a utility study. The main empirical finding is that calibrated effective rank is usually far below nominal rank, and that energy retention and statistical surprise answer different questions.
