FAITH：金融领域表格内在幻觉评估框架
FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Finance
AI总结：
针对金融表格数据中LLM幻觉评估缺失的问题，提出FAITH框架，通过掩码跨度预测任务和基于标普500报告的数据集，系统评估内在幻觉，助力构建可靠金融AI。
摘要：
幻觉仍然是大型语言模型（LLM）在金融领域部署中面临的关键挑战。从表格数据中准确提取和精确计算对于可靠的金融分析至关重要，因为即使是微小的数值错误也可能破坏决策和监管合规。金融应用具有独特需求，通常依赖上下文相关、数值型和专有的表格数据，而现有的幻觉基准很少涵盖这些数据。在本研究中，我们开发了一个严谨且可扩展的框架，用于评估金融LLM的内在幻觉，将其概念化为对真实世界金融文档的上下文感知掩码跨度预测任务。我们的主要贡献包括：（1）一种使用掩码策略的新型自动化数据集创建范式；（2）一个源自标普500年度报告的新幻觉评估数据集；（3）对最先进LLM在金融表格数据上的内在幻觉模式进行全面评估。我们的工作为内部LLM评估提供了稳健的方法论，并朝着构建更可信、更可靠的金融生成式AI系统迈出了关键一步。
英文摘要：
Hallucination remains a critical challenge for deploying Large Language Models (LLMs) in finance. Accurate extraction and precise calculation from tabular data are essential for reliable financial analysis, since even minor numerical errors can undermine decision-making and regulatory compliance. Financial applications have unique requirements, often relying on context-dependent, numerical, and proprietary tabular data that existing hallucination benchmarks rarely capture. In this study, we develop a rigorous and scalable framework for evaluating intrinsic hallucinations in financial LLMs, conceptualized as a context-aware masked span prediction task over real-world financial documents. Our main contributions are: (1) a novel, automated dataset creation paradigm using a masking strategy; (2) a new hallucination evaluation dataset derived from S&P 500 annual reports; and (3) a comprehensive evaluation of intrinsic hallucination patterns in state-of-the-art LLMs on financial tabular data. Our work provides a robust methodology for in-house LLM evaluation and serves as a critical step toward building more trustworthy and reliable financial Generative AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
