FFE-Hallu：固定修辞表达中的幻觉：波斯语中的成语和谚语基准
FFE-Hallu:Hallucinations in Fixed Figurative Expressions:Benchmark of Idioms and Proverbs in the Persian Language
AI总结：
FFE-Hallu是首个针对波斯语隐喻幻觉评估的基准，揭示了LLMs在处理隐喻语言方面的系统性弱点。
摘要：
隐喻语言，特别是固定隐喻表达（FFE）如成语和谚语，对大型语言模型（LLMs）构成了持续的挑战。与字面短语不同，FFE是文化根植的，大多非组合性的，并且通常固定，使它们特别容易受到隐喻幻觉的影响。我们定义隐喻幻觉为生成或支持那些听起来像成语且合理，但并不在目标语言中作为真实隐喻表达存在的表达。我们引入了FFEHallu，这是首个全面评估LLMs隐喻幻觉的基准，重点于波斯语，这是一种语言丰富但代表性不足的语言。FFEHallu包含600个精心挑选的实例，涵盖三个互补的任务：（i）从意义生成FFE，（ii）在四个受控构建类别中检测伪造的FFE，（iii）从英语到波斯语的FFE到FFE翻译。评估六个最先进的多语言LLMs，我们发现隐喻能力及文化根植方面存在系统性弱点。尽管像GPT4.1这样的模型在拒绝伪造的FFE和检索真实表达方面表现相对较强，但大多数模型在可靠地区分真实表达与高质量伪造表达以及在跨语言翻译中频繁产生幻觉方面存在困难。这些发现揭示了当前LLMs处理隐喻语言的显著差距，并强调了需要针对评估和减轻隐喻幻觉的基准测试的必要性。
英文摘要：
Figurative language, particularly fixed figurative expressions (FFEs) such as idioms and proverbs, poses persistent challenges for large language models (LLMs). Unlike literal phrases, FFEs are culturally grounded, largely non-compositional, and conventionally fixed, making them especially vulnerable to figurative hallucination. We define figurative hallucination as the generation or endorsement of expressions that sound idiomatic and plausible but do not exist as authentic figurative expressions in the target language. We introduce FFEHallu, the first comprehensive benchmark for evaluating figurative hallucination in LLMs, with a focus on Persian, a linguistically rich yet underrepresented language. FFEHallu consists of 600 carefully curated instances spanning three complementary tasks: (i) FFE generation from meaning, (ii) detection of fabricated FFEs across four controlled construction categories, and (iii) FFE to FFE translation from English to Persian. Evaluating six state of the art multilingual LLMs, we find systematic weaknesses in figurative competence and cultural grounding. While models such as GPT4.1 demonstrate relatively strong performance in rejecting fabricated FFEs and retrieving authentic ones, most models struggle to reliably distinguish real expressions from high quality fabrications and frequently hallucinate during cross lingual translation. These findings reveal substantial gaps in current LLMs handling of figurative language and underscore the need for targeted benchmarks to assess and mitigate figurative hallucination.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
