Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and then imitates the privileged expert conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.
EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉科学学院);Wuhan AI Research(武汉人工智能研究院)
Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Tsinghua University(清华大学);Meituan(美团);Meta AI;NiuTrans Research, Shenyang, China(新译研究院,沈阳,中国)
Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院自然语言处理实验室,中国沈阳);Pattern Recognition Center, WeChat AI, Tencent Inc, China(腾讯公司微信人工智能部门模式识别中心,中国)
Double Distillation Network for Multi-Agent Reinforcement Learning
School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);School of Automation Engineering, University of Electronic Science and Technology of China(电子科技大学自动化工程学院)
From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search
Beijing Institute of Technology(北京理工大学);East China Normal University(华东师范大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学)
TAME-Nav: Temporal Asymmetric Mixture-of-Experts Distillation for Agile and Versatile Robot Navigation
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology(东方理工大学);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学)