Large language models (LLMs) are widely applied to data analytics over documents, yet direct reasoning over long, noisy documents remains brittle and error-prone. Hence, we study document question answering (QA) that consolidates dispersed evidence into a structured output (e.g., a table, graph, or chunks) to support reliable, verifiable QA. We propose a two-pillar framework, LiteCoST, to achieve both high accuracy and low latency with small language models (SLMs). Pillar 1: Chain-of-Structured-Thought (CoST). We introduce a CoST template, a schema-aware instruction that guides a strong LLM to produce both a step-wise CoST trace and the corresponding structured output. The process induces a minimal structure, normalizes entities/units, aligns records, serializes the output, and verifies/refines it, yielding auditable supervision. Pillar 2: SLM fine-tuning. The compact models are trained on LLM-generated CoST data in two stages: Supervised Fine-Tuning for structural alignment, followed by Group Relative Policy Optimization (GRPO) incorporating triple rewards for answer/format quality and process consistency. By distilling structure-first behavior into SLMs, this approach achieves LLM-comparable quality on multi-domain long-document QA using 3B/7B SLMs, while delivering 2-4x lower latency than GPT-4o and DeepSeek-R1 (671B). The code is available at https://github.com/HKUSTDial/LiteCoST.
Distilling Mathematical Reasoning Capabilities into Small Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）
AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization
Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西）；Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西）
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）；Institute of Digital Twin, EIT（数字孪生研究院）；Saarland University（萨尔兰大学）；Meituan Inc.（美团公司）
Domain-Adapted Small Language Models for Reliable Clinical Triage
Department of Computer Science, Virginia Tech, Blacksburg, VA, US（弗吉尼亚理工学院计算机科学系，弗吉尼亚理工学院，布莱克斯堡，VA，美国）；Children’s National Hospital, Washington DC, USA（儿童医学中心，华盛顿特区，美国）
Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation
School of Computing, Informatics Institute of Technology（计算学院，信息学院技术研究所）；School of Mathematics and Computer Science, Swansea University（数学与计算机科学学院，萨默塞特大学）；R&D, Zame AI（Zame AI研发部）
Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Faculty of Engineering, Graduate Faculty of Interdisciplinary Research, University of Yamanashi（山梨大学工学部）
Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models
Zhejiang University（浙江大学）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Liangzhu Laboratory and Institute of Wenzhou, Zhejiang University（浙江大学良渚实验室与温州研究院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring
School of Computing and Information Systems, University of Melbourne, Australia（墨尔本大学计算机与信息系统学院）；School of Computer Science, University of Auckland, New Zealand（奥克兰大学计算机科学学院）
Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences（中国科学院智能信息处理重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；National University of Singapore（新加坡国立大学）
ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics
College of Design and Engineering, National University of Singapore（新加坡国立大学设计与工程学院）；Department of Mathematics, National University of Singapore（新加坡国立大学数学系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；National Science Library, Chinese Academy of Sciences（中国科学院国家科学图书馆）
MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants
South China Agricultural University（华南农业大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Electronic Science and Technology of China（电子科技大学）
Bridging the Reasoning Gap in Vietnamese with Small Language Models via Test-Time Scaling
Computer Science and Engineering（计算机科学与工程）；Information Science Faculty（信息科学系）；Faculty of Information（信息学院）；Faculty of Advanced Technologies and Engineering（先进技术和工程学院）；VNU Vietnam（越南VNU）；Japan University（日本大学）；JAIST；VNU University of Engineering and Technology（VNU工程大学）；VNU Vietnam Japan University（越南日本大学）；Hanoi, Vietnam（越南河内）；Ishikawa, Japan（日本石川）
From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；South China University of Technology（华南理工大学）
