Large language models (LLMs) such as GPT-4o and Claude Sonnet 4.5 have demonstrated strong capabilities in open-ended reasoning and generative language tasks, leading to their widespread adoption across a broad range of NLP applications. However, for structured text classification problems with fixed label spaces, model selection is often driven by predictive performance alone, overlooking operational constraints encountered in production systems.
In this work, we present a systematic comparison of two contrasting paradigms for text classification: zero- and few-shot prompt-based large language models, and fully fine-tuned encoder-only architectures. We evaluate these approaches across four canonical benchmarks (IMDB, SST-2, AG News, and DBPedia), measuring predictive quality (macro F1), inference latency, and monetary cost.
We frame model evaluation as a multi-objective decision problem and analyze trade-offs using Pareto frontier projections and a parameterized utility function reflecting different deployment regimes. Our results show that fine-tuned encoder-based models from the BERT family achieve competitive, and often superior, classification performance while operating at one to two orders of magnitude lower cost and latency compared to zero- and few-shot LLM prompting.
Overall, our findings suggest that indiscriminate use of large language models for standard text classification workloads can lead to suboptimal system-level outcomes. Instead, fine-tuned encoders emerge as robust and efficient components for structured NLP pipelines, while LLMs are better positioned as complementary elements within hybrid architectures. We release all code, datasets, and evaluation protocols to support reproducibility and cost-aware NLP system design.
State of the Art in Text Classification for South Slavic Languages: Fine-Tuning or Prompting?
Jožef Stefan Institute（乔泽夫·斯塔芬研究所）；Faculty of Computer and Information Science（计算机与信息科学系）；University of Ljubljana（卢布尔雅纳大学）；Institute of Contemporary History（当代历史研究所）
AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization
Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西）；Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西）
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
The University of Georgia（佐治亚大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Northwestern Polytechnical University（西北工业大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Mayo Clinic（梅奥诊所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models
Zhejiang University（浙江大学）；Ant Group（蚂蚁集团）；University of Liverpool（利物浦大学）；Cleveland Clinic Lerner Research Institution（克利夫兰诊所勒纳研究机构）；Soochow University（苏州大学）
Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation
Mathematical Sciences Institute, Australian National University（澳大利亚国立大学数学科学研究所）；Research School of Astronomy & Astrophysics, Australian National University（澳大利亚国立大学天文与天体物理研究学院）；School of Computing, Australian National University（澳大利亚国立大学计算学院）；Department of Astronomy, The Ohio State University（俄亥俄州立大学天文学系）
From Prompting to Preference Optimization: A Comparative Study of LLM-based Automated Essay Scoring
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam（信息科技学院，科学大学，胡志明市，越南）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学，胡志明市，越南）
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Indiana University Bloomington（印第安纳大学伯明顿分校）；Center for Complex Networks and Systems（复杂网络与系统中心）
Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
University of Trento（特伦托大学）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核子研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（斯德哥尔摩大学）
NCL-UoR at SemEval-2026 Task 5: Embedding-Based Methods, Fine-Tuning, and LLMs for Word Sense Plausibility Rating
Independent Researcher（独立研究者）；Department of Computer Science, University of Reading（阅读大学计算机科学系）；School of Computing, Newcastle University（新castle大学计算学院）
