Calibrated Adaptation: Bayesian Stiefel Manifold Priors for Reliable Parameter-Efficient Fine-Tuning
Department of Computer Science, Iowa State University（计算机科学系，爱荷华州立大学）；Department of Civil, Construction and Environmental Engineering, Iowa State University（土木、建设与环境工程系，爱荷华州立大学）
Parameter-efficient fine-tuning methods such as LoRA enable practical adaptation of large language models but provide no principled uncertainty estimates, leading to poorly calibrated predictions and unreliable behavior under domain shift. We introduce Stiefel-Bayes Adapters (SBA), a Bayesian PEFT framework that places a Matrix Langevin prior over orthonormal adapter factors on the Stiefel manifold $\St$ and performs approximate posterior inference via tangent space Laplace approximation with geodesic retraction. Unlike Gaussian priors in flat space projected onto orthogonality constraints, our prior on the manifold naturally encodes the inductive bias that adapter subspaces should be well conditioned and orthogonal, while the posterior provides calibrated predictive uncertainty without recalibration. We prove formally that the tangent space approximation strictly avoids the structural variance inflation inherent in projecting from ambient space, establishing a rigorous theoretical advantage for intrinsic manifold inference. Across GLUE and SuperGLUE benchmarks on RoBERTa-large, LLaMA-2-7B, LLaMA-2-13B, Mistral-7B, and Qwen2.5-7B, domain shift evaluations, selective prediction protocols, and an abstractive summarization task, SBA achieves task performance comparable to LoRA and DoRA while reducing Expected Calibration Error by 18 to 34\% over deterministic baselines, improving selective prediction AUROC by 12 to 25\% under domain shift, and outperforming deep ensembles of five LoRA models on OOD detection at a fraction of the parameter cost. Our results demonstrate that where you place uncertainty, on the right geometric structure, matters more than simply adding any Bayesian treatment to adapters.
Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning
Center for Data Science, Peking University（北京大学数据科学中心）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）；Beijing International Center for Mathematical Research, Peking University（北京大学北京国际数学研究中心）；Department of Mathematics, University of California, Berkeley（美国加州大学伯克利分校数学系）；Center for Machine Learning Research and Changsha Institute for Computing and Digital Economy, Peking University（北京大学机器学习研究中心和长沙计算与数字经济研究院）
Evidentially Calibrated Source-Free Time-Series Domain Adaptation with Temporal Imputation
Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）
OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（复杂系统认知与决策智能重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis
University of Copenhagen, Copenhagen, Denmark；Rutgers University, New Brunswick, NJ, USA；Section of Health Data Science \& AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark；MRC Centre for Global Infectious Disease Analysis, Department of Infectious Disease Epidemiology, School of Public Health, Faculty of Medicine, Imperial College London, London, United Kingdom
Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data
College of Computing and Data Science, Nanyang Technological University (NTU), Singapore（南洋理工大学计算与数据科学学院）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I2R），A*STAR，新加坡）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息科学技术大学人工智能学院）
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校（UIUC））；Imperial College London（伦敦帝国学院）
FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation
Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai（人工智能与未来网络研究院，北京师范大学，珠海）；School of Computer Science and Artificial Intelligence, Zhengzhou University（计算机科学与人工智能学院，郑州大学）；School of Software, Nanchang University（软件学院，南昌大学）
Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting
School of Software Engineering, South China University of Technology（华南理工大学软件工程学院）；Key Laboratory of Big Data and Intelligent Robot, Ministry of Education（教育部大数据与智能机器人重点实验室）；Pazhou Laboratory（琶洲实验室）；Nanyang Technological University（南洋理工大学）；ByteDance, China（字节跳动（中国））；Tencent AI Lab（腾讯AI实验室）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）
Unsupervised Calibration through Prior Adaptation for Text Classification using Large Language Models
Instituto de Investigación en Ciencias de la Computación, CONICET-UBA（国家科学技术研究委员会-布宜诺斯艾利斯大学计算机科学研究所）；Departamento de Electrónica, Facultad de Ingeniería, Universidad de Buenos Aires（布宜诺斯艾利斯大学工程学院电子系）；Centro de Simulación Computacional para Aplicaciones Tecnológicas, CONICET（国家科学技术研究委员会技术应用计算模拟中心）；Laboratoire des Signaux et Systèmes (L2S), CentraleSupelec CNRS Universite Paris-Saclay（巴黎萨克雷大学中央理工-高等电力学院法国国家科学研究中心信号与系统实验室（L2S））
