How Likely Do LLMs with CoT Mimic Human Reasoning?
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究院)
Chain-of-thought emerges as a promising technique for eliciting reasoning capabilities from Large Language Models (LLMs). However, it does not always improve task performance or accurately represent reasoning processes, leaving unresolved questions about its usage. In this paper, we diagnose the underlying mechanism by comparing the reasoning process of LLMs with humans, using causal analysis to understand the relationships between the problem instruction, reasoning, and the answer in LLMs. Our empirical study reveals that LLMs often deviate from the ideal causal chain, resulting in spurious correlations and potential consistency errors (inconsistent reasoning and answers). We also examine various factors influencing the causal structure, finding that in-context learning with examples strengthens it, while post-training techniques like supervised fine-tuning and reinforcement learning on human feedback weaken it. To our surprise, the causal structure cannot be strengthened by enlarging the model size only, urging research on new techniques. We hope that this preliminary study will shed light on understanding and improving the reasoning process in LLM.
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人机混合增强智能国家重点实验室、视觉信息与应用国家工程研究中心、人工智能与机器人研究所);Nankai University(南开大学);The Hong Kong University of Science and Technology(Guangzhou)(香港科技大学(广州));School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院)
Where Do CoT Training Gains Land in LLM based Agents?
ByteDance(字节跳动);Beijing Key Laboratory of Research on Large Models and Intelligent Governance(北京市大模型与智能治理重点实验室);Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE(教育部新一代智能搜索与推荐工程研究中心)
COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion
School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院);Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(中国电子科学与技术大学深圳高级研究所);School of Computer Science, The University of Sydney(悉尼大学计算机科学学院)
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative(宁波空间智能与数字衍生关键实验室);Institute of Digital Twin, EIT(数字孪生研究院);Saarland University(萨尔兰大学);Meituan Inc.(美团公司)
Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notes
Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children's Hospital of Philadelphia(费城儿童医院雷蒙德·G·佩雷尔曼细胞与分子治疗中心);University of Pennsylvania(宾夕法尼亚大学);Perelman School of Medicine, University of Pennsylvania(宾夕法尼亚大学佩雷尔曼医学院)
Cognitive-Mental-LLM: Evaluating Reasoning in Large Language Models for Mental Health Prediction via Online Text
Ira A. Fulton Schools of Engineering Arizona State University Tempe, USA(伊拉·A·福林工程学校亚利桑那州立大学Tempe分校);Department of Psychology Santa Clara University Santa Clara, USA(心理学系圣克拉拉大学Santa Clara分校)
Knowledge Graph Augmented Large Language Models for Disease Prediction
Department of Computer Science, Emory University(埃默里大学计算机科学系);Division of Medical Sciences, Oxford University(牛津大学医学系);Department of Computer Science and Engineering, UCSC(UCSC计算机科学与工程系);Nell Hodgson Woodruff School of Nursing, Emory University(埃默里大学内尔·霍根·伍德鲁夫护理学院);School of Medicine, Emory University(埃默里大学医学院);Rollins School of Public Health, Emory University(埃默里大学罗林斯公共卫生学院)
PRISMA: Preference-Reinforced Self-Training Approach for Interpretable Emotionally Intelligent Negotiation Dialogues
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India(印度理工学院帕纳瓦分校计算机科学与工程系);School of Artificial Intelligence and Data Science, Indian Institute of Technology Jodhpur, India(印度理工学院朱道尔分校人工智能与数据科学学院)
Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
University of Illinois Chicago(伊利诺伊大学芝加哥分校);Google Research(谷歌研究);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Politecnico di Milano(米兰理工学院)
Efficient Reasoning via Chain of Unconscious Thought
Huazhong University of Science and Technology(华中科技大学);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Moonshot AI(月之暗面);Zhejiang Normal University(浙江师范大学)