缓解大语言模型中的地理空间知识幻觉：基准测试与动态事实对齐
Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning
AI总结：
该研究针对大语言模型的地理空间知识幻觉问题，构建了基于知识图谱的评估框架并测试20个模型，提出基于KTO的动态事实对齐方法，在基准上实现超29.6%的性能提升，增强了模型地理空间任务的可信度。
摘要：
大语言模型（LLM）拥有广泛的世界知识，包括地理空间知识，已成功应用于移动性预测、社会指标预测等各类地理空间任务。然而，LLM 常生成不准确的地理空间知识，引发地理空间幻觉（即地理空间信息的错误或不一致表征），损害其可靠性。尽管 LLM 的通用知识幻觉现象已被广泛研究，但地理空间幻觉的系统性评估与缓解仍鲜有探索。为填补这一空白，我们提出一套全面的地理空间幻觉评估框架，利用结构化地理空间知识图谱实现可控评估。通过对 20 个先进 LLM 的广泛评估，我们揭示了它们地理空间知识中的幻觉问题。基于这些发现，我们提出一种基于卡尼曼-特沃斯基优化（Kahneman-Tversky Optimization, KTO）的动态事实对齐方法，以缓解 LLM 的地理空间幻觉，在所提基准上实现了超过 29.6% 的性能提升。大量实验结果表明，我们的基准与学习算法在提升 LLM 在地理空间知识与推理任务中的可信度方面具有有效性。
英文摘要：
Large language models (LLMs) possess extensive world knowledge, including geospatial knowledge, which has been successfully applied to various geospatial tasks such as mobility prediction and social indicator prediction. However, LLMs often generate inaccurate geospatial knowledge, leading to geospatial hallucinations (incorrect or inconsistent representations of geospatial information) that compromise their reliability. While the phenomenon of general knowledge hallucination in LLMs has been widely studied, the systematic evaluation and mitigation of geospatial hallucinations remain largely unexplored. To address this gap, we propose a comprehensive evaluation framework for geospatial hallucinations, leveraging structured geospatial knowledge graphs for controlled assessment. Through extensive evaluation across 20 advanced LLMs, we uncover the hallucinations in their geospatial knowledge. Building on these insights, we introduce a dynamic factuality aligning method based on Kahneman-Tversky Optimization (KTO) to mitigate geospatial hallucinations in LLMs, leading to a performance improvement of over 29.6% on the proposed benchmark. Extensive experimental results demonstrate the effectiveness of our benchmark and learning algorithm in enhancing the trustworthiness of LLMs in geospatial knowledge and reasoning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
