CoT-Driven Framework for Short Text Classification: Enhancing and Transferring Capabilities from Large to Smaller Model
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院)
Hui Wu, Yuanben Zhang, Zhonghe Han, Yingyan Hou, Lei Wang, Siye Liu, Qihang Gong, Yunping Ge
Short Text Classification (STC) is crucial for processing and understanding the brief but substantial content prevalent on contemporary digital platforms. The STC encounters difficulties in grasping the semantic and syntactic intricacies, an issue that is apparent in traditional pre-trained language models. Although Graph Convolutional Networks enhance performance by integrating external knowledge bases, these methods are limited by the quality and extent of the knowledge applied. Recently, the emergence of Large Language Models (LLMs) and Chain-of-Thought (CoT) has significantly improved the performance of complex reasoning tasks. However, some studies have highlighted the limitations of their application in fundamental NLP tasks. Consequently, this study first employs CoT to investigate and enhance the capabilities of LLMs in STC tasks. We propose the Syntactic and Semantic Enrichment CoT (SSE-CoT) method, effectively decomposing the STC tasks into four distinct steps: (i) essential concept identification, (ii) common-sense knowledge retrieval, (iii) text rewriting, and (iv) classification. Furthermore, recognizing resource constraints in sectors like finance and healthcare, we then introduce the CoT-Driven Multi-Task Learning (CDMT) framework to extend these capabilities to smaller models. This framework begins by extracting rationales from LLMs and subsequently fine-tunes smaller models to optimize their performance. Extensive experimentation across six short-text benchmarks validated the efficacy of the proposed methods. In particular, SSE-CoT achieved state-of-the-art performance with substantial improvements on all datasets, particularly on the Ohsumed and TagMyNews datasets.
CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs
School of Computer Science & Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(计算机科学与工程学院,新一代人工智能技术及其交叉应用关键实验室(东南大学),教育部,中国);Data Science & AI Department at Faculty of IT, Monash University, Australia(信息学院数据科学与人工智能部门,墨尔本大学,澳大利亚)
Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学人工智能研究院计算机科学与技术系);School of Computer Science, Beijing University of Posts and Telecommunications, China(北京邮电大学计算机科学学院)
CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis
School of Information, Renmin University of China(中国人民大学信息学院);Tsinghua University(清华大学);Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China(数据工程与知识工程重点实验室)
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA(约翰·F·肯尼迪高中弗里蒙特分校);Queensland Academy for Science Mathematics and Technology(昆士兰科学数学与技术学院);Research Spark Hub Inc(Research Spark Hub公司)
SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(区块链与数据安全国家重点实验室,浙江大学);State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学)
Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
CoT2Align: Cross-Chain of Thought Distillation via Optimal Transport Alignment for Language Models with Different Tokenizers
Hanoi University of Science and Technology(河内理工大学);Oraichain Labs Inc.(Oraichain实验室公司);ByteDance Inc(字节跳动公司);VNU University of Engineering and Technology(越南国家大学工程技术大学);Monash University(莫纳什大学);University of Oregon(俄勒冈大学)
CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China(华中科技大学计算机科学与技术学院);Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China(华中科技大学光电实验室);Department of Computer Science, Bowling Green State University, Ohio, USA(Bowling Green State University 计算机科学系);Department of Computer Science, Aalborg University, Aalborg, Denmark(Aalborg University 计算机科学系);Huawei Technologies Co., Ltd, Shenzhen, China(华为技术有限公司)
Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines
University of Pittsburgh(匹兹堡大学);University of Pittsburgh Medical Center(匹兹堡大学医学中心);University of Pittsburgh Medical Center Western Maryland(匹兹堡大学医学中心西马里兰分院)
Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN
The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(认知与决策智能复杂系统实验室,自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Baichuan Inc(百川公司);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
FairCoT: Enhancing Fairness in Text-to-Image Generation via Chain of Thought Reasoning with Multimodal Large Language Models
School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦女王学院电子工程与计算机科学学院);Department of Knowledge Technologies, Jožef Stefan Institute(Jožef Stefan研究所知识技术系)
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
Understanding Before Reasoning: Enhancing Chain-of-Thought with Iterative Summarization Pre-Prompting
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science(电子工程学院,上海工程技术大学);School of Information Science and Engineering, Ningbo University(信息科学与工程学院,宁波大学)