Eliciting explicit, step-by-step reasoning traces from large language models (LLMs) has emerged as a dominant paradigm for enhancing model capabilities. Although such reasoning strategies were originally designed for problems requiring explicit multi-step reasoning, they have increasingly been applied to a broad range of NLP tasks. This expansion implicitly assumes that deliberative reasoning uniformly benefits heterogeneous tasks. However, whether such reasoning mechanisms truly benefit classification tasks remains largely underexplored, especially considering their substantial token and time costs. To fill this gap, we introduce TextReasoningBench, a systematic benchmark designed to evaluate the effectiveness and efficiency of reasoning strategies for text classification with LLMs. We compare seven reasoning strategies, namely IO, CoT, SC-CoT, ToT, GoT, BoC, and long-CoT across ten LLMs on five text classification datasets. Beyond traditional metrics such as accuracy and macro-F1, we introduce two cost-aware evaluation metrics that quantify the performance gain per reasoning token and the efficiency of performance improvement relative to token cost growth. Experimental results reveal three notable findings: (1) Reasoning does not universally improve classification performance: while moderate strategies such as CoT and SC-CoT yield consistent but limited gains (typically +1% to +3% on big models), more complex methods (e.g., ToT and GoT) often fail to outperform simpler baselines and can even degrade performance, especially on small models; (2) Reasoning is often inefficient: many reasoning strategies increase token consumption by 10$\times$ to 100$\times$ (e.g., SC-CoT and ToT) while providing only marginal performance improvements.
Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist
Xi’an Jiaotong-Liverpool University(西交利物浦大学);University of Liverpool(利物浦大学);University of Macau(澳门大学);HKUST(香港科技大学);Microsoft Research Asia(微软亚洲研究院);Duke Kunshan University(昆山杜克大学)
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Microsoft Research(微软研究院);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);CUHKSZ(香港中文大学(深圳));THU(清华大学)
Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering(陕西省大数据知识工程重点实验室);National Engineering Lab for Big Data Analytics(国家大数据分析工程实验室);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
"I know myself better, but not really greatly": How Well Can LLMs Detect and Explain LLM-Generated Texts?
School of Cyber Science and Engineering, Shanghai Jiao Tong University, China(上海交通大学信息科学与工程学院);Department of Computing Science, University of Aberdeen, UK(阿伯丁大学计算机科学系);Institute of Cyber Security for Society (iCSS) & School of Computing, University of Kent, UK(社会网络安全研究所(iCSS)及肯特大学计算机学院)
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
University of Glasgow(格拉斯哥大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Artificial Intelligence, Shandong University(山东大学人工智能学院)
Do Efficient Transformers Really Save Computation?
School of EECS, Peking University(北京大学信息与工程科学学院(电子工程与计算机科学学院));ETH Zürich(苏黎世联邦理工学院);National Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学通用人工智能重点实验室、智能科学与技术学院);New York University(纽约大学);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院);Center for Machine Learning Research, Peking University(北京大学机器学习研究中心)
An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai Innovation Center for Processor Technologies(上海处理器技术创新中心);School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学与技术学院)
What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code
State Key Laboratory of Cognitive Intelligence, University of Science(认知智能国家重点实验室,科学大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Centerce(人工智能研究院,合肥综合性国家科学中心);Individual Researcher(个人研究员);Zhejiang University, Hangzhou, China(浙江大学,杭州,中国)
Internal Consistency and Self-Feedback in Large Language Models: A Survey
School of Information, Renmin University of China(中国人民大学信息学院);Large Language Model Center, Institute for Advanced Algorithms Research(高级算法研究院大语言模型中心);School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);State Key Laboratory of Media Convergence Production Technology and Systems, Xinhua News Agency(新华社媒体融合生产技术与系统国家重点实验室)
Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?
University of Delhi(德里大学);Jamia Hamdard(贾迈亚哈马尔德大学);Cornell University(康奈尔大学);Expedia Group(Expedia集团);DSEU-Okhla;Center for SDGC(SDGC中心);Stanford University(斯坦福大学)
What's the Meaning of Superhuman Performance in Today's NLU?
Babelscape(巴别斯克);Sapienza University of Rome(罗马第一大学);University of Groningen(格罗宁根大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Charles University(查理大学);University of Copenhagen(哥本哈根大学);University of Melbourne(墨尔本大学);Carnegie Mellon University(卡内基梅隆大学);Saarland University(萨尔大学);Jožef Stefan Institute(约泽夫·斯特凡研究所);University of Ljubljana(卢布尔雅那大学);Cardiff University(卡迪夫大学);University of Zurich(苏黎世大学);University of Edinburgh(爱丁堡大学);University of Amsterdam(阿姆斯特丹大学)
Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);Alibaba Group(阿里巴巴集团)
Can Large Language Model Agents Simulate Human Trust Behavior?
KAUST(阿卜杜拉国王科技大学);Illinois Institute of Technology(伊利诺伊理工大学);Pennsylvania State University(宾夕法尼亚州立大学);University of Chicago(芝加哥大学);Emory(埃默里大学);University of Oxford(牛津大学);California Institute of Technology(加州理工学院);University of Michigan(密歇根大学);Santa Fe Institute(圣达菲研究所);Google(谷歌公司)
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?
University of Chicago(芝加哥大学);NUS(新加坡国立大学);Stanford University(斯坦福大学);UCSC(加州大学圣克鲁兹分校);Duke University(杜克大学);ESSEC(埃塞克高等商学院);Illinois Tech(伊利诺伊理工学院);UCSD(加州大学圣地亚哥分校);Peking University(北京大学);University of Queensland(昆士兰大学);Stony Brook University(石溪大学);UNC-Chapel Hill(北卡罗来纳大学教堂山分校);USTC(中国科学技术大学)