大型语言模型在音乐学中的角色:我们准备好信任机器了吗?
The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?
AI总结:
本文探讨了大型语言模型在音乐学中的可靠性与接受度,提出基于检索增强生成和人工验证的半自动基准构建方法,并发现普通LLMs不如基于音乐词典的检索增强生成可靠,强调需要领域知识驱动的专业化研究。
摘要:
在这项工作中,我们探讨了大型语言模型(LLMs)在音乐学中的使用及其可靠性。基于与专家和学生的讨论,我们评估了当前对这一如今无处不在的技术的接受程度和担忧。我们旨在更进一步,提出一种半自动方法,利用检索增强生成模型和多项选择题生成来创建一个初始基准,并由人类专家进行验证。我们在400道经人类验证的问题上的评估表明,当前的普通LLMs不如基于音乐词典的检索增强生成可靠。本文表明,LLMs在音乐学中的潜力需要以音乐学为导向的研究,通过纳入准确可靠的领域知识来使LLMs专业化。
英文摘要:
In this work, we explore the use and reliability of Large Language Models (LLMs) in musicology. From a discussion with experts and students, we assess the current acceptance and concerns regarding this, nowadays ubiquitous, technology. We aim to go one step further, proposing a semi-automatic method to create an initial benchmark using retrieval-augmented generation models and multiple-choice question generation, validated by human experts. Our evaluation on 400 human-validated questions shows that current vanilla LLMs are less reliable than retrieval augmented generation from music dictionaries. This paper suggests that the potential of LLMs in musicology requires musicology driven research that can specialized LLMs by including accurate and reliable domain knowledge.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。