发表机构
Charles University; Institute of Formal and Applied Linguistics（查理大学; 形式与应用语言学研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究大语言模型对输入上下文（事实、反事实、虚构）的忠实度，发现上下文-记忆冲突较弱，且评判者选择影响冲突强度评估。
AI 中文摘要
大语言模型（LLMs）容易产生幻觉或误解事实，这损害了它们在检索增强生成或数据到文本系统中的可用性。我们分析了LLMs对给定上下文的忠实度如何取决于它们对该上下文合理性的感知（上下文-记忆冲突）。为了更好地识别错误模式，我们利用了非英语和低资源语言文本生成以及基于局部知识的输入数据（这些数据仅部分包含在模型的参数化知识中）所增加的难度。我们让模型从包含捷克和斯洛伐克本地数据的事实性（FA）、反事实性（CFA）和虚构性（FI）RDF三元组中生成英语、捷克语、斯洛伐克语和上索布语文本。与我们的预期相反，我们在人工标注的样本上仅观察到微弱的上下文-记忆冲突。对于作为LLM评判者的Kimi K3（其与人工标注在样本上高度一致），反事实输入获得的忠实度分数仅略低于事实性输入（在1-5分制上相差-0.05）。我们还发现，LLM评判者的次优选择会导致高估上下文-记忆冲突的强度。
英文摘要
Large language models (LLMs) are prone to hallucinating or misinterpreting facts, which impairs their usability in retrieval-augmented generation or data-to-text systems. We analyse how faithfulness of LLMs to provided context depends on how plausible they perceive the context to be (context-memory conflict). To better identify error patterns, we make use of the increased difficulty of non-English and low-resource language text generation and input data based on local knowledge, only partially captured in models' parametric knowledge. We let the models generate text in English, Czech, Slovak and Upper Sorbian from factual (FA), counterfactual (CFA) and fictional (FI) RDF triples containing local Czech and Slovak data. Contrary to our expectations, we observe only a weak context-memory conflict on the human-annotated sample. For Kimi K3 as an LLM judge, which agrees well with human annotations on the sample, counterfactual inputs receive only slightly lower faithfulness scores than factual ones (-0.05 on a 1-5 scale). We also find that a suboptimal choice of LLM judge would lead to overestimating the strength of the context-memory conflict.
Comments16 pages, 2 figures, to be published in INLG 2026
