Large language models (LLMs) can solve problems step-by-step. While this chain-of-thought (CoT) reasoning boosts LLMs' performance, it is unclear if LLMs \textit{know} when to use CoT and whether those CoT are always necessary to answer the question. This paper shows that LLMs tend to generate redundant calculations and reasoning on a manually constructed math QA dataset, GSM8K-Zero. GSM8K-Zero is constructed such that the questions can be answered without any calculations, but LLMs, including Llama-2 models and Claude-2, tend to generate lengthy and unnecessary calculations to answer the questions. We also conduct experiments to explain why LLMs generate redundant calculations and reasonings. GSM8K-Zero is publicly available at https://github.com/d223302/Over-Reasoning-of-LLMs and https://huggingface.co/datasets/dcml0714/GSM8K-Zero.
Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteria
POSTECH;Seoul National University(首尔国立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Kyungpook National University(庆北国立大学);LG AI Research(LG AI研究院)
SEEK: Steering LLM Reasoning for RAG via Internal Reasoning Sketches
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学计算机科学与技术系,人工智能研究院);School of Intelligent Science and Technology, Nanjing University, China(南京大学智能科学与技术学院)
Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Department of Computer Science, National University of Singapore(新加坡国立大学计算机科学系);Ant Group(蚂蚁集团)
SyncThink: A Training-Free Strategy to Align Inference Termination with Reasoning Saturation
National Key Laboratory for Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家级重点实验室);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);School of Computer Science, Peking University(北京大学计算机科学学院)
On the Token Value Inequality in Efficient Reasoning
Purdue University(普渡大学);Rochester Institute of Technology(罗切斯特理工学院);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);University of Missouri-Kansas City(密苏里大学堪萨斯城分校);Rutgers University(罗格斯大学);Meta AI
ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy
National Key Laboratory for Multimedia Information Processing, Peking University(国家多媒体信息处理重点实验室,北京大学);School of Software and Microelectronics, Peking University(软件与微电子学院,北京大学);School of Computer Science, Peking University(计算机学院,北京大学)
Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models
University of Electronic Science and Technology of China(电子科技大学);Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province(四川省 ubiquitous 智能与可信服务重点实验室)
THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models
School of Artificial intelligence, JiLin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, JiLin University(吉林大学知识驱动人机智能工程研究中心);International Center of Future Science, JiLin University(吉林大学未来科学国际中心)
FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse
School of ASEE, Beihang University(北京航空航天大学ASEE学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);School of BME, Beijing Advanced Innovation Center for BME, Beihang University(北京航空航天大学生物医学工程学院/北京生物医学工程高精尖创新中心);CAIR and CECS, VinUniversity(VinUniversity计算与人工智能研究中心/计算机工程与计算机科学系)
Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learning
Renmin University of China(中国人民大学);Xiaohongshu Inc.(小红书公司);Institute of Automation,Chinese Academy of Sciences(中国科学院自动化研究所);University of Southern California(南加州大学);Shandong University(山东大学)
Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Artificial Intelligence and Data Science, USTC(中国科学技术大学人工智能与数据科学学院);Kuaishou Technology(快手科技);College of Future Information Technology, Fudan University(复旦大学未来信息技术学院)
Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models
Department of Computer Science and Engineering, POSTECH, South Korea(计算机科学与工程系,POSTECH,韩国);Graduate School of Artificial Intelligence, POSTECH, South Korea(人工智能研究生院,POSTECH,韩国)
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
University of Science and Technology of China(科学技术大学);Shenzhen University of Advanced Technology(深圳先进技术大学);Shenzhen Institutes of Advanced Technology, CAS(深圳先进技术研究所,中国科学院);Southeast University(东南大学);Nanjing University(南京大学);Beihang University(北航);University of Manchester(曼彻斯特大学)
InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
Institute for Infocomm Research (I 2 R), A*STAR, Singapore(信息与通信研究机构(I 2 R),A*STAR,新加坡);Centre for Frontier AI Research (CFAR), A*STAR, Singapore(前沿人工智能研究中心(CFAR),A*STAR,新加坡)
Self-signals Driven Multi-LLM Debate for Efficient and Accurate Reasoning
University of Cambridge(剑桥大学);Sorbonne Université(索邦大学);University of Science and Technology of China(中国科学技术大学);Beihang University(北航大学);Nanyang Technological University(南洋理工大学)