Chain-of-thought (CoT) rationales, which provide step-by-step reasoning to derive final answers, benefit LLMs in both inference and training. Incorporating rationales, either by generating them before answering during inference, or by placing them before or after the original answers during training - significantly improves model performance on mathematical, symbolic and commonsense reasoning tasks. However, most work focuses on the role of rationales in these reasoning tasks, overlooking their potential impact on other important tasks like natural language understanding (NLU) tasks. In this work, we raise the question: Can rationales similarly benefit NLU tasks? To conduct a systematic exploration, we construct NLURC, a comprehensive and high-quality NLU dataset collection with rationales, and develop various rationale-augmented methods. Through exploring the applicability of these methods on NLU tasks using the dataset, we uncover several potentially surprising findings: (1) CoT inference shifts from hindering NLU performance to surpassing direct label prediction as model size grows, indicating a positive correlation. (2) Most rationale-augmented training methods perform worse than label-only training, with one specially designed method consistently achieving improvements. (3) LLMs trained with rationales achieve significant performance gains on unseen NLU tasks, rivaling models ten times their size, while delivering interpretability on par with commercial LLMs.
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Computer Science, University of Illinois at Chicago(芝加哥大学计算机科学系)
Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancements
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(一般人工智能国家重点实验室,智能科学与技术学院,北京大学)
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources
DAMO Academy, Alibaba Group(达摩院(阿里巴巴集团));Nanyang Technological University(南洋理工大学);Singapore University of Technology and Design(新加坡科技设计大学);Salesforce Research(Salesforce研究院);Hupan Lab(湖畔实验室)
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting
Institute of Artificial Intelligence and Future Networks, Beijing Normal University(北京师范大学人工智能与未来网络研究院);Hong Kong Baptist University(香港浸会大学);BNU-HKBU United International College(北京师范大学-香港浸会大学联合国际学院);College of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning
Michigan State University(密歇根州立大学);Amazon(亚马逊);Northeastern University(东北大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Northwestern University(西北大学)
Free-text Rationale Generation under Readability Level Control
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Computational Linguistics, Department of Linguistics, Universität Potsdam(波茨坦大学语言学系计算语言学部门);Quality and Usability Lab, Technische Universität Berlin(柏林技术大学质量与可用性实验室);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院);Computer Science Institute, Hochschule Ruhr West(鲁尔西部大学计算机科学研究所)
Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification
University of Warwick(沃里克大学);Queen Mary University of London(伦敦大学女王学院);University of Bologna(博洛尼亚大学);King’s College London(伦敦大学国王学院);The Alan Turing Institute(艾伦·图灵研究所)
Knowledge-Augmented Multimodal Clinical Rationale Generation for Disease Diagnosis with Small Language Models
Hong Kong Baptist University(香港 Baptist 大学);Shanxi University(山西大学);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海先进研究院);Hong Kong University of Science and Technology(香港科技大学);The University of Manchester(曼彻斯特大学)
CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China(上海交通大学计算机科学学院);Shanghai KEY Laboratory of Integrated Administration Technologies for Information Security, Shanghai, China(上海信息安全集成管理技术重点实验室);School of Politics, National Defense University, Shanghai, China(国防大学政治学院)
From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment
Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Lund University(吕勒奥大学);University of Tübingen(图宾根大学);Stanford Graduate School of Education(斯坦福大学教育研究生院);Harvard Graduate School of Education(哈佛大学教育研究生院)