Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
Ubiquitous Knowledge Processing Lab (UKP Lab), TU Darmstadt and ATHENE National Research Center for Applied Cybersecurity, Germany(图宾根大学知识处理实验室(UKP Lab)及德国应用网络安全国家研究中心ATHENE);Dept. of ECE & Ingenuity Labs Research Institute, Queen’s University, Canada(皇后大学电子工程系及创新实验室研究机构);University of Bath, UK(巴斯大学)
Requiring a large language model (LLM) to generate intermediary reasoning steps, known as Chain of Thought (CoT), has been shown to be an effective way of boosting performance. Previous approaches have focused on generating multiple independent CoTs, combining them through ensembling or other post-hoc strategies to enhance reasoning. In this work, we introduce a novel approach where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step, which is fundamentally different from prior work that primarily operate on parallel CoT generations. DCoT allows LLMs to gain the ability to perform within-inference refinement of reasoning chains without requiring external feedback. Through a rigorous set of experiments spanning a wide range of tasks that require various reasoning types, we show that fine-tuning on DCoT improves performance over the CoT baseline across model families and scales (1.3B to 70B). These improvements are particularly impactful for tasks with a large result state space, such as those involving numeric answers. Our work is also significant because both quantitative analyses and manual evaluations reveal the observed gains stem from the models' ability to refine an initial reasoning chain by generating a second, improved chain within the same inference step, demonstrating previously elusive self-improvement. Our code and data are publicly available at https://github.com/UKPLab/acl2025-diverse-cot.
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaohongshu Inc(小红书公司)
Unlocking Recursive Thinking of LLMs: Alignment via Refinement
Soochow University(苏州大学);Zhipu AI(智谱AI);Tsinghua University(清华大学);Key Laboratory of Data Intelligence and Advanced Computing, Soochow University(苏州大学数据智能与先进计算重点实验室)
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
University of Science and Technology of China(科学技术大学);Shenzhen University of Advanced Technology(深圳先进技术大学);Shenzhen Institutes of Advanced Technology, CAS(深圳先进技术研究所,中国科学院);Southeast University(东南大学);Nanjing University(南京大学);Beihang University(北航);University of Manchester(曼彻斯特大学)
Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
School of Engineering, Westlake University(西湖大学工程学院);City University of Hong Kong(香港城市大学);Research Center for Industries of the Future, Westlake University(西湖大学未来产业研究中心);Johns Hopkins University(约翰斯·霍普金斯大学)
MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants
South China Agricultural University(华南农业大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Electronic Science and Technology of China(电子科技大学)
CoT-Driven Framework for Short Text Classification: Enhancing and Transferring Capabilities from Large to Smaller Model
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院)
Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes
Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children's Hospital of Philadelphia(费城儿童医院雷蒙德·G·佩雷尔曼细胞与分子治疗中心);University of Pennsylvania(宾夕法尼亚大学);Perelman School of Medicine, University of Pennsylvania(宾夕法尼亚大学佩雷尔曼医学院)
Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Meituan(美团);University of Science and Technology of China(中国科学技术大学)
CoEvoT: Co-Evolving Chain-of-Thought Prompting for Graph-LLM Reasoning
Sun Yat-Sen University(中山大学);The Chinese University of Hong Kong(香港中文大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);National University of Singapore(新加坡国立大学);University of Electronic Science and Technology of China(电子科技大学);Beijing University of Posts and Telecommunieations(北京邮电大学);Singapore Management University(新加坡管理大学)
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement
University of Manchester(曼彻斯特大学);Idiap Research Institute(Idiap研究所);National Biomarker Centre, CRUK-MI, University of Manchester(曼彻斯特大学英国癌症研究中心曼彻斯特研究所国家生物标志物中心)
Cognitive-Mental-LLM: Evaluating Reasoning in Large Language Models for Mental Health Prediction via Online Text
Ira A. Fulton Schools of Engineering Arizona State University Tempe, USA(伊拉·A·福林工程学校亚利桑那州立大学Tempe分校);Department of Psychology Santa Clara University Santa Clara, USA(心理学系圣克拉拉大学Santa Clara分校)
TDA-RC: Task-Driven Alignment for Knowledge-Based Reasoning Chains in Large Language Models
School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);College of Professional and Continuing Education, The Hong Kong Polytechnic University(香港理工大学专业及持续教育学院);School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)机电工程与自动化学院);School of Computing, Kyung Hee University(庆熙大学计算机学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)
KGQuest: Template-Driven QA Generation from Knowledge Graphs with LLM-Based Refinement
Scuola Superiore Sant’Anna, Pisa, Italy(圣安娜高等学院);Sapienza University of Rome, Rome, Italy(罗马萨皮恩扎大学);Institute of Informatics and Telematics, National Research Council of Italy (CNR)(意大利信息与电信研究所,国家研究理事会(CNR))
CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China(华中科技大学计算机科学与技术学院);Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China(华中科技大学光电实验室);Department of Computer Science, Bowling Green State University, Ohio, USA(Bowling Green State University 计算机科学系);Department of Computer Science, Aalborg University, Aalborg, Denmark(Aalborg University 计算机科学系);Huawei Technologies Co., Ltd, Shenzhen, China(华为技术有限公司)
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Saarland Informatics Campus(萨尔兰信息学园区);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心(CERTAIN));Technische Universität Berlin(柏林工业大学);IBM Research(IBM研究院);University of Copenhagen(哥本哈根大学);University of Groningen(格罗宁根大学)