Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.
Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation
LIVIA, Dept. of Systems Engineering, ETS Montreal(LIVIA,系统工程系,蒙特利尔工程学院);CentraleSupélec, Universite Paris Saclay(中央理工巴黎高等学院);Aiim Lab, Queen’s University(人工智能实验室,皇后大学)
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Security Capability Center(360安全能力中心);Nanjing University(南京大学);Tongji University(同济大学);Shanghai Jiao Tong University(上海交通大学)
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Tsinghua University(清华大学);Meituan(美团);Meta AI;NiuTrans Research, Shenyang, China(新译研究院,沈阳,中国)
Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
The University of Hong Kong(香港大学);Simon Fraser University(西蒙菲莎大学);University of California, Los Angeles(加州大学洛杉矶分校);Dartmouth College(达特茅斯学院);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Tencent(腾讯)
Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents
Qwen Large Model Application Team, Alibaba(阿里巴巴通义千问大模型应用团队);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学);Shanghai Jiao Tong University(上海交通大学)
Distilling Lightweight Domain Experts from Large ML Models by Identifying Relevant Subspaces
Konrad Zuse School of Excellence in Learning;BIFOLD\,--\,Berlin Institute for the Foundations of Learning;Department of Artificial Intelligence, Korea University, Seoul 136-713, Korea;Max Planck Institute for Informatics, 66123 Saarbr \"u cken, Germany;Institute for AI in Medicine, Charit\'e\,--\,Universit\"atsmedizin Berlin, 10115 Berlin, Germany
Learning in the Fisher Subspace: A Guided Initialization for LoRA Fine-Tuning
Department of Computer Science(计算机科学系);Information Engineering, National Cheng Kung University(信息工程系,国立成功大学);Department of Artificial Intelligence, Chang Gung University(人工智能系,长庚大学);Department of Computer Science, National Tsing Hua University(计算机科学系,国立清华大学)
DeSyR: A decoupled symbolic recovery framework for PDEs with PINN-guided structure discovery and physics-based coefficient refinement
Chengdu University of Information Technology(成都信息工程大学);Sichuan University(四川大学);Hong Kong Polytechnic University(香港理工大学);Southwest Petroleum University(西南石油大学)
University of Tübingen(图宾根大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);ELLIS Institute Tübingen(图宾根ELLIS研究所);Tübingen AI Center(图宾根人工智能中心)
Mutual Information Guided Backdoor Mitigation for Pre-trained Encoders
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算机与数据科学学院)
Lightweight Intrusion Detection in IoT via SHAP-Guided Feature Pruning and Knowledge-Distilled Kronecker Networks
LSIA Laboratory, Moroccan School of Engineering Sciences (EMSI)(摩洛哥工程科学学校LSIA实验室);Laboratory of Research in Informatics (LaRI), Faculty of Sciences, Ibn Tofail University(伊本·泰菲勒大学信息研究实验室);College of Computing, University Mohammed VI Polytechnic(穆莱·伊沙克六世理工学院计算机学院)
From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology(北京智能信息科技重点实验室,计算机科学与技术学院,北京理工大学);ByteDance China(字节跳动中国)
On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Large Language Model Department, Tencent(腾讯大语言模型部);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);National University of Singapore(新加坡国立大学);Wuhan AI Research(武汉人工智能研究院)