发表机构
School of Artificial Intelligence, Jilin University; Michigan State University（吉林大学人工智能学院; 密歇根州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出DALorRA，一种变分贝叶斯稀疏框架，通过随机掩码低秩适应中的秩维度实现模型容量正则化和校准，在不牺牲推理精度下提升LLM校准性能。
AI 中文摘要
大型语言模型（LLMs）展现出卓越的推理能力，但其任务特定微调常因过度自信而严重阻碍可信部署。我们提出数据自适应低秩适应（DALorRA），一个简单有效的变分贝叶斯稀疏框架，将不确定性量化的范式从密集参数空间转移到低秩适应（LoRA）的轻量级秩级别。基于LoRA本质上聚合了多个可能提供多余模型容量的秩一分量的洞察，DALorRA对秩维度施加随机掩码，从而在训练期间实现模型容量的贝叶斯正则化，在推理期间实现类似集成的校准。大量实验表明，DALorRA在不牺牲推理精度的情况下实现了LLMs的出色校准。
英文摘要
Large language models (LLMs) exhibit remarkable reasoning capabilities, but their task-specific fine-tuning is notoriously plagued by overconfidence, severely hindering trustworthy deployment. We propose Data-Adaptive Lower-Rank Adaptation (DALorRA), a simple and effective variational Bayesian sparse framework that shifts the paradigm of uncertainty quantification from the dense parameter space to the lightweight rank level of low-rank adaptation (LoRA). With the insight that LoRA essentially aggregates multiple rank-one components that may provide superfluous model capacity, DALorRA imposes stochastic masking on rank dimensions, enabling Bayesian regularization of model capacity during training and ensemble-like calibration during inference. Extensive experiments demonstrate DALorRA's excellent calibration of LLMs without compromising reasoning accuracy.
CommentsTo appear in EMNLP 2026. 17 pages, 7 figures, 8 tables
