arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11295cs.CL
收藏

基于大语言模型的时序文本分类

Temporal Text Classification with Large Language Models

George Mason University(乔治·马歇尔大学)

Nishat Raihan, Marcos Zampieri

AI总结:

本研究评估了多种大语言模型在时序文本分类任务中的表现,发现专有模型在少样本提示下表现优异,而微调虽能提升开源模型性能,但仍无法超越专有模型。

摘要:

语言会随时间变化。计算模型可以训练以识别这种变化,从而使它们能够估计文本的出版日期。尽管近年来在大语言模型(LLMs)方面取得了进展,但它们在自动 dating 文本,也称为时序文本分类(TTC)方面的表现尚未被探索。本研究首次系统评估了领先的专有(Claude 3.5,GPT-4o,Gemini 1.5)和开源(LLaMA 3.2,Gemma 2,Mistral,Nemotron 4)LLMs 在使用三个历史语料库(其中两个为英文,一个为葡萄牙语)上的TTC表现。我们测试了零样本和少样本提示以及微调设置。我们的结果表明,专有模型表现良好,尤其是在少样本提示下。它们还表明,微调显著提高了开源模型的性能,但这些模型仍无法达到专有LLMs所提供的性能。

英文摘要:

Languages change over time. Computational models can be trained to recognize such changes enabling them to estimate the publication date of texts. Despite recent advancements in Large Language Models (LLMs), their performance on automatic dating of texts, also known as Temporal Text Classification (TTC), has not been explored. This study provides the first systematic evaluation of leading proprietary (Claude 3.5, GPT-4o, Gemini 1.5) and open-source (LLaMA 3.2, Gemma 2, Mistral, Nemotron 4) LLMs on TTC using three historical corpora, two in English and one in Portuguese. We test zero-shot and few-shot prompting, and fine-tuning settings. Our results indicate that proprietary models perform well, especially with few-shot prompting. They also indicate that fine-tuning substantially improves open-source models but that they still fail to match the performance delivered by proprietary LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑