arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.12152cs.CL
收藏

用大型语言模型重新审视自动化主题模型评估

Revisiting Automated Topic Model Evaluation with Large Language Models

ETH Zürich(苏黎世联邦理工学院);University of Maryland(马里兰大学)

Dominik Stammbach, Vilém Zouhar, Alexander Hoyle, Mrinmaya Sachan, Elliott Ash

AI总结:

本文提出利用大型语言模型评估主题模型输出,发现其与人类判断相关性优于现有指标,并可通过为文档自动分配标签来确定最佳主题数量。

摘要:

主题模型用于理解大型文本集合。然而,自动评估主题模型输出并确定最佳主题数量一直是长期存在的挑战,迄今尚无有效的自动化解决方案。本文提出使用大型语言模型来评估此类输出。我们发现,大型语言模型能恰当地评估生成的主题,与现有自动化指标相比,其与人类判断的相关性更强。随后,我们调查了能否使用大型语言模型自动确定最佳主题数量。我们自动为文档分配标签,并选择具有最纯标签的配置,从而为最佳主题数量返回合理的值。

英文摘要:

Topic models are used to make sense of large text collections. However, automatically evaluating topic model output and determining the optimal number of topics both have been longstanding challenges, with no effective automated solutions to date. This paper proposes using large language models to evaluate such output. We find that large language models appropriately assess the resulting topics, correlating more strongly with human judgments than existing automated metrics. We then investigate whether we can use large language models to automatically determine the optimal number of topics. We automatically assign labels to documents and choosing configurations with the most pure labels returns reasonable values for the optimal number of topics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑