Large language models (LLMs) with chain-of-thought reasoning achieve state-of-the-art performance across complex problem-solving tasks, but their verbose reasoning traces and large context requirements make them impractical for edge deployment. These challenges include high token generation costs, large KV-cache footprints, and inefficiencies when distilling reasoning capabilities into smaller models for mobile devices. Existing approaches often rely on distilling reasoning traces from larger models into smaller models, which are verbose and stylistically redundant, undesirable for on-device inference. In this work, we propose a lightweight approach to enable reasoning in small LLMs using LoRA adapters combined with supervised fine-tuning. We further introduce budget forcing via reinforcement learning on these adapters, significantly reducing response length with minimal accuracy loss. To address memory-bound decoding, we exploit parallel test-time scaling, improving accuracy at minor latency increase. Finally, we present a dynamic adapter-switching mechanism that activates reasoning only when needed and a KV-cache sharing strategy during prompt encoding, reducing time-to-first-token for on-device inference. Experiments on Qwen2.5-7B demonstrate that our method achieves efficient, accurate reasoning under strict resource constraints, making LLM reasoning practical for mobile scenarios. Videos demonstrating our solution running on mobile devices are available on our project page.
When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo(宁波数字孪生研究院,东方理工高等研究院,宁波);Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院);The Hong Kong Polytechnic University(香港理工大学);LMU Munich(慕尼黑大学);Saarland University(萨尔大学)
LiveThinking: Enabling Real-Time Efficient Reasoning for AI-Powered Livestreaming via Reinforcement Learning
Taobao \& Tmall Group of Alibaba Hangzhou Zhejiang China;Taobao \& Tmall Group of Alibaba Beijing China;ROLL Team of Alibaba Hangzhou Zhejiang China;Taobao \& Tmall Group of Alibaba;ROLL Team of Alibaba
Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning
Beihang University(北航);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));National University of Defense Technology(国防科技大学);Hong Kong Polytechnic University(香港理工大学);Alibaba Group(阿里巴巴集团)
What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis
Ningbo Institute of Digital Twin, Eastern Institute of Technology(宁波数字孪生研究院,东方理工大学);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系)
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Zhongguancun Academy(中关村学院)
Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLM
University of Southern California(南加州大学);The University of Queensland(昆士兰大学);University of California, San Diego(加州大学圣地亚哥分校);University of California, Merced(加州大学默塞德分校)
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Baidu Inc.(百度公司);Tencent Inc.(腾讯公司)
InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
Institute for Infocomm Research (I 2 R), A*STAR, Singapore(信息与通信研究机构(I 2 R),A*STAR,新加坡);Centre for Frontier AI Research (CFAR), A*STAR, Singapore(前沿人工智能研究中心(CFAR),A*STAR,新加坡)
JudgeRLVR: Judge First, Generate Second for Efficient Reasoning
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);CFCS, School of Computer Science, Peking University(计算机学院,北京大学);LLM-Core Xiaomi(小智LLM核心)
LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
University of Texas at Dallas(德克萨斯大学达拉斯分校);Emory University(埃默里大学);Individual Researcher(独立研究员);University of Texas at Arlington(德克萨斯大学阿灵顿分校);University of Florida(佛罗里达大学)