arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18045cs.CL
收藏

Multi-FAct:使用 FActScore 评估多语言大语言模型的事实性

Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore

School of Computing(计算机学院);KAIST(韩国科学技术院)

Sheikh Shafayat, Eunsu Kim, Juhyun Oh, Alice Oh

AI总结:

本文提出 Multi-FAct 流程,将 FActScore 扩展到多语言场景,评估多语言大语言模型在区域多样性主题长文本生成中的事实性,并给出基于非英语维基百科的评估指南。

摘要:

评估长文本大语言模型(LLM)生成内容的事实性是一项重要挑战。近期,针对英语的事实性评估研究迅速增多,但对于多语言大语言模型的事实性评估,尤其是长文本生成场景下的评估,仍知之甚少。我们引入了一个简单的多语言事实性评估流程,通过将 FActScore(Min 等,2023)应用于多种语言来实现。除了评估多语言事实性生成之外,我们还评估了反映区域多样性主题的长文本生成的事实准确性。我们还考察了使用非英语维基百科运行 FActScore 流程的可行性,并针对区域多样性主题的多语言事实性评估提供了全面指南。

英文摘要:

Evaluating the factuality of long-form large language model (LLM)-generated text is an important challenge. Recently there has been a surge of interest in factuality evaluation for English, but little is known about the factuality evaluation of multilingual LLMs, specially when it comes to long-form generation. %This paper systematically evaluates multilingual LLMs' factual accuracy across languages and geographic regions. We introduce a simple pipeline for multilingual factuality evaluation, by applying FActScore (Min et al., 2023) for diverse languages. In addition to evaluating multilingual factual generation, we evaluate the factual accuracy of long-form text generation in topics that reflect regional diversity. We also examine the feasibility of running the FActScore pipeline using non-English Wikipedia and provide comprehensive guidelines on multilingual factual evaluation for regionally diverse topics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑