Large language models (LLMs) achieve state-of-the-art accuracy on complex reasoning tasks by generating multiple chain-of-thought (CoT) traces, but using a fixed token budget per query leads to over-computation on easy inputs and under-computation on hard ones. We introduce Predictive Scheduling, a plug-and-play framework that pre-runs lightweight predictors, an MLP on intermediate transformer hidden states or a LoRA-fine-tuned classifier on raw question text, to estimate each query's optimal reasoning length or difficulty before any full generation. Our greedy batch allocator dynamically distributes a fixed total token budget across queries to maximize expected accuracy. On the GSM8K arithmetic benchmark, predictive scheduling yields up to 7.9 percentage points of absolute accuracy gain over uniform budgeting at identical token cost, closing over 50\% of the gap to an oracle with perfect foresight. A systematic layer-wise study reveals that middle layers (12 - 17) of the transformer carry the richest signals for size estimation. These results demonstrate that pre-run budget prediction enables fine-grained control of the compute-accuracy trade-off, offering a concrete path toward latency-sensitive, cost-efficient LLM deployments.
MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);The International Joint Institute of Tianjin University, Tianjin University(天津大学国际联合学院);Faculty of Digital Economics and Managements, Tianjin University of Finance and Economics(天津财经大学数字经济与管理学院)
Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning
Key Laboratory of Multimedia Trusted Perception(多媒体可信感知实验室);Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China(教育部高效计算实验室,厦门大学,361005,中国);Sino-Russian ResearchCenter for Digital Economy(中俄数字经济研究中心);Institute of Artificial Intelligence, Xiamen University, China(人工智能研究院,厦门大学,中国);School of Informatics, Xiamen University, China(信息学院,厦门大学,中国);School of Information Engineering, Xiamen Ocean Vocational College, Xiamen 361102, China(信息工程学院,厦门海洋职业技术学院,厦门361102,中国)
The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);School of Fashion and Textiles, The Hong Kong Polytechnic University(香港理工大学纺织及制衣学院);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院)
Evaluating the Efficacy of LLM-Based Reasoning for Multiobjective HPC Job Scheduling
University of Tennessee, Knoxville\ Ridge National Laboratory Oak Ridge, TN USA;Argonne National Laboratory Lemont, IL USA;University of Southern California Los Angeles, CA USA;Oak Ridge National Laboratory Oak Ridge, TN USA;University of Tennessee, Knoxville\ Ridge National Laboratory;Argonne National Laboratory;University of Southern California;Oak Ridge National Laboratory
Non-myopic Generation of Language Models for Reasoning and Planning
The University of Hong Kong(香港大学);Peking University(北京大学);Zhejiang University(浙江大学);Westlake University(西湖大学);The Hong Kong University of Science and Technology(香港科技大学)
Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Cranberry-Lemon University(Cranberry-Lemon 大学);University of the Witwatersrand(独立研究者);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control
University of Michigan(密歇根大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Dalle Molle Institute for Artificial Intelligence Research (IDSIA)(达勒莫尔人工智能研究所);Università della Svizzera italiana (USI)(瑞士意大利语区大学);Scuola universitaria professionale della Svizzera italiana (SUPSI)(瑞士意大利语区应用科学与艺术大学)
LLMs-based Few-Shot Disease Predictions using EHR: A Novel Approach Combining Predictive Agent Reasoning and Critical Agent Instruction
Emory University(埃默里大学);University of Washington(华盛顿大学);UCSD(加利福尼亚大学圣迭戈分校);Rollins School of Public Health, Emory University(埃默里大学罗林斯公共卫生学院);School of Medicine, Emory University(埃默里大学医学院)
Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling
School of Computer Science and Engineering, Beihang University, Beijing 100191, China(北京航空航天大学计算机科学与工程学院);Shenzhen Loop Area Institute, Shenzhen, China(深圳环形区研究所);Qingdao Research Institute, Beihang University(青岛研究院);Hangzhou Innovation Institute, Beihang University(杭州创新研究院);School of Artificial Intelligence, Xidian University, Xi’an 710071, Shaanxi, China(西安电子科技大学人工智能学院);Guangzhou Institute of Technology, Xidian University, Guangzhou 510555, Guangdong, China(广州技术研究所)
A Multi-Level Validation and Traceability Framework for AI-Generated Telescope Scheduling Decisions
Yunnan Observatories, Chinese Academy of Sciences(云南天文台,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Key Laboratory of the Structure and Evolution of Celestial Objects, Chinese Academy of Sciences(中国科学院天文结构与演化重点实验室);Yunnan Key Laboratory of Solar Physics and Space Science(云南太阳物理与空间科学重点实验室);Center for Astronomical Mega-Science, Chinese Academy of Sciences(中国科学院天文大科学中心)
ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
CFAR, Agency for Science, Technology and Research (A*STAR), Singapore(新加坡科技研究局CFAR);The Hong Kong University of Science and Technology, Hong Kong(香港科技大学);Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));Hong Kong Baptist University, Hong Kong(香港浸会大学);Nanyang Technological University, Singapore(南洋理工大学);The Hong Kong University of Science and Technology (Guangzhou), China(香港科技大学(广州))
Predictive Reasoning with Augmented Anomaly Contrastive Learning for Compositional Visual Relations
School of Computer Science, University of Nottingham Ningbo China(诺丁汉大学宁波校区计算机科学学院);Institute of Biomedical Engineering, Ningbo institute of materials technology and engineering, Chinese Academy of Sciences(中国科学院宁波材料技术与工程研究所生物医学工程研究所);School of Electrical & Electronic Engineering, Nanyang Technological University(南洋理工大学电子与电气工程学院)
Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge Graphs
School of Computer Science(计算机科学学院);Wuhan University(武汉大学);School of Computer Science and Informatics(计算机科学与信息学院);Cardiff University(卡迪夫大学);College of Information Science and Engineering(信息科学与工程学院);Shanxi Agricultural University(山西农业大学);School of Artificial Intelligence(人工智能学院);Center for Language and Information Research(语言与信息研究中心);University of Manchester(曼彻斯特大学);ILCC, School of Informatics(信息学院);University of Edinburgh(爱丁堡大学)