arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.24060cs.CLcs.AI
收藏

任务复杂性至关重要:对LLM在情感分析中推理能力的实证研究

Task Complexity Matters: An Empirical Study of Reasoning in LLMs for Sentiment Analysis

School of Computing and Information Systems, Singapore Management University(新加坡管理学院);Research and Development, Mastercard(麦star公司)

Donghao Huang, Zhaoxia Wang

AI总结:

研究通过评估7种模型家族在不同粒度的情感分析数据集上的表现,发现推理效果高度依赖任务复杂性,二分类任务性能下降,多类情感识别性能提升,且少量样本学习在多数情况下优于零样本学习。

摘要:

大型语言模型(LLMs)具备推理能力,推动了推理普遍提升语言任务性能的理论。本文通过评估504种配置,测试了推理在情感分析任务中的有效性。研究发现,推理效果强烈依赖任务复杂性:二分类任务性能下降达-19.9 F1百分点,而27类情感识别性能提升达+16.0百分点;精简推理变体在简单任务中表现欠佳,但少量样本提示可部分恢复;少量样本学习在多数情况下优于零样本学习,收益因架构和任务复杂性而异;帕累托前沿分析显示基础模型在效率-性能权衡中占优,推理仅在复杂情感识别中合理,尽管计算开销高2.1倍至54倍。定量结果辅以定性错误分析,揭示推理通过系统性过度推理降低简单任务性能,提供超越高层过度思考假设的机制洞察。

英文摘要:

Large language models (LLMs) with reasoning capabilities have fueled a compelling narrative that reasoning universally improves performance across language tasks. We test this claim through a comprehensive evaluation of 504 configurations across seven model families--including adaptive, conditional, and reinforcement learning-based reasoning architectures--on sentiment analysis datasets of varying granularity (binary, five-class, and 27-class emotion). Our findings reveal that reasoning effectiveness is strongly task-dependent, challenging prevailing assumptions: (1) Reasoning shows task-complexity dependence--binary classification degrades up to -19.9 F1 percentage points (pp), while 27-class emotion recognition gains up to +16.0pp; (2) Distilled reasoning variants underperform base models by 3-18 pp on simpler tasks, though few-shot prompting enables partial recovery; (3) Few-shot learning improves over zero-shot in most cases regardless of model type, with gains varying by architecture and task complexity; (4) Pareto frontier analysis shows base models dominate efficiency-performance trade-offs, with reasoning justified only for complex emotion recognition despite 2.1x-54x computational overhead. We complement these quantitative findings with qualitative error analysis revealing that reasoning degrades simpler tasks through systematic over-deliberation, offering mechanistic insight beyond the high-level overthinking hypothesis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑