Deploying LLM agents at scale typically requires choosing between quality and cost. Existing cost-reduction approaches fail to preserve agility: the ability to iterate rapidly without human time bottlenecks. Prompt engineering is brittle and slows iteration, while fine-tuning requires multi-day training and commitment to fixed designs; both are impractical for iterative workflows and time-sensitive batch jobs. We demonstrate that established inference-time techniques--dynamic in-context learning and self-consistency cascades--can be leveraged to shift the cost-accuracy Pareto frontier while preserving agility. Practitioners run the teacher on a small task subset to collect demonstrations, then immediately deploy a cheaper student on the remainder. At each step, the system retrieves relevant teacher demonstrations as in-context examples. When multiple student samples agree, we proceed; when they diverge, we fall back to the teacher. This requires no prompt engineering or training. On ALFWorld, we match teacher accuracy at 2.5x lower cost (0.059 to 0.024 per episode). On AppWorld, we achieve 3.5x cost reduction while recovering 79% of teacher accuracy. Our empirical analyses provide guidance on key design choices: teacher database size, demonstration set size, retrieval strategy, and cascade thresholds. These analyses highlight inference-time levers for navigating cost-performance tradeoffs without sacrificing human development speed.
AI can evolve without labels: self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation
KAIST(韩国科学技术院);Asan Medical Center(峨山医学中心);University of Ulsan College of Medicine(蔚山大学医学院);Yeungnam University(岭南大学);Kyungpook National University(庆北国立大学);Seoul National University(首尔国立大学)
MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
Department of Computer Science and Engineering, Southern University of Science and Technology(计算机科学与工程系,南方科技大学);School of Mathematical and Computer Sciences, University of Adelaide(数学与计算机科学学院,阿德莱德大学)
Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
University of Chinese Academy of Sciences(中国科学院大学);University of South Florida(佛罗里达州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Key Laboratory of System Software (Chinese Academy of Sciences)(中国科学院系统软件重点实验室);State Key Laboratory of Computer Science (Institute of Software, Chinese Academy of Sciences)(中国科学院软件研究所计算机科学国家重点实验室)
LT-PINN: Lagrangian Topology-conscious Physics-informed Neural Network for Boundary-focused Engineering Optimization
Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hong Kong, China(土木与环境工程系,香港理工大学,香港,中国);Research Institute for Sustainable Urban Development (RISUD), The Hong Kong Polytechnic University, Hong Kong, China(可持续城市发展研究院(RISUD),香港理工大学,香港,中国);Department of Mechanical Engineering, The Hong Kong Polytechnic University, Hong Kong, China(机械工程系,香港理工大学,香港,中国);School of Fashion and Textiles, The Hong Kong Polytechnic University, Hong Kong, China(时尚与纺织学院,香港理工大学,香港,中国);Department of Mechanical Engineering, The University of Hong Kong, Hong Kong, China(机械工程系,香港大学,香港,中国)