TailNLG：一种针对长尾实体 verbalization 的多语言基准
TailNLG: A Multilingual Benchmark Addressing Verbalization of Long-Tail Entities
AI总结：
本文提出TailNLG基准，评估大型语言模型在长尾实体上的表现，揭示其存在偏见，强调需更可靠的评估框架。
摘要：
结构化知识的自动 verbalization 是使知识图谱对非专家用户可访问并支持检索增强生成系统的关键任务。尽管数据到文本生成的最新进展提高了多语言覆盖，但长尾实体的潜在偏见却很少受到关注。本文首次系统研究了数据到文本生成中的长尾实体。我们引入了TailNLG，一个新多语言基准，涵盖英文、意大利语和西班牙语，基于Wikidata，覆盖不同流行程度的实体。我们评估了三种不同家族的大规模语言模型在零样本设置下的表现，并比较了它们在罕见与常见实体上的性能，以及与已建立的WebNLG基准的比较。我们的结果揭示了长尾实体的一致偏见：基于嵌入的分数较低，模型不确定性较高。我们进一步表明，长尾实体的影响在不同模型和语言中有所不同，现有评估指标无法一致捕捉这些差异，突显了需要更可靠评估框架的必要性。
英文摘要：
The automatic verbalization of structured knowledge is a key task for making knowledge graphs accessible to non-expert users and supporting retrieval-augmented generation systems. Although recent advances in Data-to-Text generation have improved multilingual coverage, little attention has been paid to potential biases in the verbalization of rare entities, frequently known as long-tail entities. In this work, we present the first systematic study of long-tail entities in Data-to-Text generation. We introduce TailNLG, a new multilingual benchmark in English, Italian, and Spanish, built from Wikidata and covering entities with varying levels of popularity. We evaluate three different families of large language models in zero-shot settings and compare their performance on rare versus common entities, as well as against the established WebNLG benchmark. Our results reveal a consistent bias against long-tail entities: embedding-based scores are lower, and model uncertainty is higher for rare entities. We further show that the impact of long-tail entities varies across models and languages, and that existing evaluation metrics do not consistently capture these differences, highlighting the need for more reliable evaluation frameworks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
