面向文本分类的大语言模型：案例研究与综合综述
Large Language Models For Text Classification: Case Study And Comprehensive Review
AI总结：
本研究在多分类和二分类两种文本分类场景下，对比评估不同规模、架构的大语言模型与传统最优模型的性能及推理时间权衡，发现提示策略显著影响模型表现，复杂任务中Llama3、GPT-4等LLMs更优，简单二分类任务中传统机器学习模型的性价比更高。
摘要：
挖掘大语言模型（Large Language Models, LLMs）在数据分类中的潜力是自然语言处理领域一个前景广阔的方向。本研究在两种不同的分类场景下，将不同LLMs的性能与当前最优的深度学习和机器学习模型进行对比评估：一是基于在线发布的工作评价对员工工作地点进行分类（多分类任务），二是对新闻文章进行真伪分类（二分类任务）。我们的分析涵盖了规模、量化方式和架构各异的多种语言模型，探究了不同提示技术的影响，并以加权F1值为指标评估模型性能。此外，我们还考察了每类语言模型在性能（F1值）与时间（推理响应时间）之间的权衡，以更细致地理解各模型的实际适用性。研究发现，提示策略的不同会导致模型响应出现显著差异。结果表明，LLMs尤其是Llama3和GPT-4，在多分类这类复杂分类任务中表现优于传统方法，但代价是推理时间更长；而更简单的机器学习模型在简单的二分类任务中具备更优的性能-时间权衡。
英文摘要：
Unlocking the potential of Large Language Models (LLMs) in data classification represents a promising frontier in natural language processing. In this work, we evaluate the performance of different LLMs in comparison with state-of-the-art deep-learning and machine-learning models, in two different classification scenarios: i) the classification of employees' working locations based on job reviews posted online (multiclass classification), and 2) the classification of news articles as fake or not (binary classification). Our analysis encompasses a diverse range of language models differentiating in size, quantization, and architecture. We explore the impact of alternative prompting techniques and evaluate the models based on the weighted F1-score. Also, we examine the trade-off between performance (F1-score) and time (inference response time) for each language model to provide a more nuanced understanding of each model's practical applicability. Our work reveals significant variations in model responses based on the prompting strategies. We find that LLMs, particularly Llama3 and GPT-4, can outperform traditional methods in complex classification tasks, such as multiclass classification, though at the cost of longer inference times. In contrast, simpler ML models offer better performance-to-time trade-offs in simpler binary classification tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
