大型语言模型表现出强大的推理能力,但往往依赖于表面模式匹配和答案记忆等捷径而非真正的逻辑推理。我们提出了Shortcut-Aware Reasoning Training (SART),一种梯度感知框架,通过ShortcutScore和梯度手术检测并缓解促进捷径的样本。我们的方法通过梯度与验证目标的不一致性和答案标记集中性来识别捷径信号,并相应修改训练动态。在受控推理基准上的实验表明,SART在最强基线基础上实现了+16.5%的准确率和+40.2%的鲁棒性,显著提高了在分布偏移下的泛化能力。代码可在:https://github.com/fuyanjie/short-cut-aware-data-centric-reasoning 获取。
英文摘要:
Large language models exhibit strong reasoning capabilities, yet often rely on shortcuts such as surface pattern matching and answer memorization rather than genuine logical inference. We propose Shortcut-Aware Reasoning Training (SART), a gradient-aware framework that detects and mitigates shortcut-promoting samples via ShortcutScore and gradient surgery. Our method identifies shortcut signals through gradient misalignment with validation objectives and answer-token concentration, and modifies training dynamics accordingly. Experiments on controlled reasoning benchmarks show that SART achieves +16.5% accuracy and +40.2% robustness over the strongest baseline, significantly improving generalization under distribution shifts. Code is available at: https://github.com/fuyanjie/short-cut-aware-data-centric-reasoning.
Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models
Department of Computer Science(计算机科学系);School of Nursing(护理学院);Department of Information Sciences and Technology(信息科学与技术系);Department of Biomedical Informatics & Medical Education(生物医学信息学与医学教育系)
Mitigating Shortcuts in Language Models with Soft Label Encoding
Stevens Institute of Technology(史蒂文斯理工学院);Shanghai Jiao Tong University(上海交通大学);New Jersey Institute of Technology(新泽西理工学院);University of Georgia(佐治亚大学)
Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering
College of Zhiyuan, Shanghai Jiao Tong University(上海交通大学致远学院);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);Shanghai Jiao Tong University(上海交通大学)
Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems
Electronic Information School, Wuhan University(武汉大学电子信息学院);School of Computing, Dublin City University(都柏林城市大学计算机学院);School of Public Health, Wuhan University(武汉大学公共卫生学院);Insight Centre for Data Analytics, University of Galway(Galway大学数据分析研究中心);Insight Centre for Data Analytics, Dublin City University(都柏林城市大学数据分析研究中心)
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Cyberspace Security, Southeast University(东南大学网络空间安全学院)
C2PO: Diagnosing and Disentangling Bias Shortcuts in LLMs
Jinan University(济南大学);Nanyang Technological University(南洋理工大学);Engineering Research Center of Trustworthy AI (Ministry of Education)(可信人工智能工程研究中心);Guangxi Key Laboratory of Trusted Software(广西可信软件重点实验室)
DR.GAP: Mitigating Bias in Large Language Models using Gender-Aware Prompting with Decoupled Reasoning
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);School of Computer and Cyber Sciences, Augusta University(奥古斯塔大学计算机与网络科学学院)
Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning
Hong Kong University of Science and Technology(香港科技大学);University of Southern California(南加州大学);Tencent AI Lab(腾讯AI实验室);University of California, Davis(加州大学戴维斯分校)
Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models
University of Science and Technology of China(中国科学技术大学);State Key Lab of Cognitive Intelligence(认知智能国家重点实验室);School of Computer Science and Technology(计算机科学与技术学院)
NTS-CoT: Mitigating Hallucinations in LLM-based News Timeline Summarization with Chain-of-Thought Reasoning
Central South University(中南大学);Tsinghua University(清华大学);Nanjing University(南京大学);Suzhou Aerospace Information Research Institute(苏州空天信息研究院);McGill University(麦吉尔大学)
Investigating the Influence of Prompt-Specific Shortcuts in AI Generated Text Detection
Seoul National University(首尔大学);Hanyang University(汉阳大学);Ewha Womans University(梨花女子大学);NAVER Search US(NAVER美国搜索部门);NAVER AI LAB(NAVER AI实验室);NAVER Cloud(NAVER云)
Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computing Technologies, RMIT University(皇家墨尔本理工学院计算技术学院)
Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(认知与决策智能复杂系统重点实验室,自动化研究所,中国科学院,北京,中国);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学);Tsinghua University(清华大学)
Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulation
Tongji University(同济大学);Hong Kong Polytechnic University(香港理工大学);Shanghai University of Electric and Power(上海电力大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy(广东省人工智能与数字经济实验室);University of Electronic Science and Technology of China(电子科技大学)