arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07003cs.CLcs.AI
收藏

生成式AI用于自动主题标注

Generative AI for automatic topic labelling

School of Library and Information Science, University of Montreal, Canada(蒙特利尔大学图书馆与信息科学学院)

Diego Kozlowski, Carolina Pradier, Pierre Benz

AI总结:

本文利用生成式AI模型对科学领域主题进行自动标注,评估了三种LLM在标注准确性与复杂性把握上的表现,发现三词标签更适用于捕捉研究主题的复杂性。

摘要:

主题建模已成为研究科学领域的重要工具,因为它能够对研究趋势进行大规模解读。然而,这些模型的输出是关键词列表,需要手动解释以进行标注。本文提出了评估三种LLM在标注可靠性方面的研究,即flan、GPT-4o和GPT-4 mini。基于先前利用BERTopic的研究,我们从所有瑞士生物学教授(n=465)在2008至2020年间发表的所有科学文章(n=34,797)中生成主题,这些文章记录在Web of Science数据库中。我们从定量和定性两个方面评估了三种模型的输出,并发现,首先,GPT模型能够准确且精确地对模型输出的关键词进行标注。其次,三词标签比二词标签更能把握研究主题的复杂性。

英文摘要:

Topic Modeling has become a prominent tool for the study of scientific fields, as they allow for a large scale interpretation of research trends. Nevertheless, the output of these models is structured as a list of keywords which requires a manual interpretation for the labelling. This paper proposes to assess the reliability of three LLMs, namely flan, GPT-4o, and GPT-4 mini for topic labelling. Drawing on previous research leveraging BERTopic, we generate topics from a dataset of all the scientific articles (n=34,797) authored by all biology professors in Switzerland (n=465) between 2008 and 2020, as recorded in the Web of Science database. We assess the output of the three models both quantitatively and qualitatively and find that, first, both GPT models are capable of accurately and precisely label topics from the models' output keywords. Second, 3-word labels are preferable to grasp the complexity of research topics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑