Evaluating the factuality of long-form large language model (LLM)-generated text is an important challenge. Recently there has been a surge of interest in factuality evaluation for English, but little is known about the factuality evaluation of multilingual LLMs, specially when it comes to long-form generation. %This paper systematically evaluates multilingual LLMs' factual accuracy across languages and geographic regions. We introduce a simple pipeline for multilingual factuality evaluation, by applying FActScore (Min et al., 2023) for diverse languages. In addition to evaluating multilingual factual generation, we evaluate the factual accuracy of long-form text generation in topics that reflect regional diversity. We also examine the feasibility of running the FActScore pipeline using non-English Wikipedia and provide comprehensive guidelines on multilingual factual evaluation for regionally diverse topics.
Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers
MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Copenhagen(哥本哈根大学);University of California, Santa Barbara(加州大学圣巴巴拉分校);Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院)
OpenFActScore: Open-Source Atomic Evaluation of Factuality in Text Generation
Philipps-Universität Marburg(马尔堡大学);German Research Centre for Artificial Intelligence (DFKI)(德国人工智能研究中心);Centre for European Research in Trusted AI (CERTAIN)(可信AI欧洲研究中心);Saarland University(萨尔兰大学)
Deconstructing Stereotypes: Scope-Conditioned Generation for Effective Multilingual Counterspeech
Université Côte d’Azur(蔚蓝海岸大学);CNRS(法国国家科学研究中心);INRIA(法国国家信息与自动化研究所);I3S(I3S实验室);PRHLT Research Center, Universitat Politècnica de València(瓦伦西亚理工大学PRHLT研究中心);ValgrAI Valencian Graduate School and Research Network of Artificial Intelligence(ValgrAI瓦伦西亚人工智能研究生院与研究网络)
Universitat Politècnica de València(瓦伦西亚理工大学);University of Trento(特伦托大学);University of Sheffield(谢菲尔德大学);Bonn-Aachen International Center for Information Technology, University of Bonn(波恩-亚琛信息科技国际中心,波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);University of Pisa(比萨大学);University of Milano-Bicocca(米兰-比可卡大学);ValgrAI Valencian Graduate School and Research Network of Artificial Intelligence(瓦伦西亚人工智能研究生学校和研究网络)
Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain
CSIR-Central Scientific Instruments Organisation, India(印度CSIR-中央科学仪器组织);Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, U.P, India(印度科学与创新研究院(AcSIR));School of Mathematical and Computational Sciences, Massey University, New Zealand(梅西大学数学与计算科学学院)
Confabulations from ACL Publications (CAP): A Dataset for Scientific Hallucination Detection
Charles University, Prague(查尔斯大学,布拉格);Independent Researcher(独立研究者);University of Helsinki, Finland(赫尔辛基大学);University of British Columbia, Vancouver, Canada(不列颠哥伦比亚大学,温哥华,加拿大);Chosun University, South Korea(成均馆大学,韩国)