How can small-scale large language models (LLMs) efficiently utilize the supervision of LLMs to improve their generative quality? This question has been well studied in scenarios where there is no restriction on the number of LLM supervisions one can use, giving birth to many decoding algorithms that utilize supervision without further training. However, it is still unclear what is an effective strategy under the $\textit{limited supervision}$ scenario, where we assume that no more than a few tokens can be generated by LLMs. To this end, we develop an algorithm to effectively aggregate the small-scale LLM and LLM predictions on initial tokens so that the generated tokens can more accurately condition the subsequent token generation by small-scale LLM only. Critically, we find that it is essential to adaptively overtrust or disregard the LLM prediction based on the confidence of the small-scale LLM. Through our experiments on a wide range of models and datasets, we demonstrate that our method provides a consistent improvement over conventional decoding strategies. $\small$ $\textbf{Code:}$ https://github.com/HJ-Ok/DecLimSup
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
University of Washington（华盛顿大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Southern California（南加州大学）；Independent Researcher（独立研究者）；National University of Singapore（新加坡国立大学）；Microsoft（微软）；Google（谷歌）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Northwestern University（西北大学）；Allen Institute for AI (AI2)（人工智能研究院（AI2））
"The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；Nankai University（南开大学）；The Hong Kong University of Science and Technology(Guangzhou)（香港科技大学（广州））；School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（软件学院，人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）
Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation
Ant Group（蚂蚁集团）；Alibaba International Digital Commerce Group（阿里巴巴国际数字商业集团）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；University of Electronic Science and Technology of China（电子科技大学）
BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception
HFIPS, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；University of Science and Technology of China（中国科学技术大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Science and Technology of China（中国科学技术大学）；Alibaba Group（阿里巴巴集团）；Stanford University（斯坦福大学）
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Tsinghua University（清华大学）；Meituan（美团）；Meta AI；NiuTrans Research, Shenyang, China（新译研究院，沈阳，中国）
Enhancing Trustworthy Clinical Diagnosis Decision-Making in Large Language Models via Etiology-Aware Attention Supervision
Zhejiang University（浙江大学）；University of Chinese Academy of Sciences（中国科学院大学）；First Affiliated Hospital, College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
University of Chinese Academy of Sciences（中国科学院大学）；Chinese Information Processing Laboratory（中文信息处理实验室）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation
University of Science and Technology of China（中国科学技术大学）；Tencent（腾讯）；Tsinghua University（清华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Zhongguancun Academy（中关村学院）；Nankai University（南开大学）；National University of Singapore（新加坡国立大学）
SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillation
University of Science and Technology of China（中国科学技术大学）；Large Language Model Department, Tencent（腾讯大语言模型部门）；Shanghai Innovation Institute（上海创新研究院）；Zhongguancun Academy（中关村学院）
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Security Capability Center（360安全能力中心）；Nanjing University（南京大学）；Tongji University（同济大学）；Shanghai Jiao Tong University（上海交通大学）
