当金融微调失效：领域适配语言模型中数值幻觉的三级可检测性分析
When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models
浏览论文内容
中文总结 AI 辅助
该研究分析金融领域适配语言模型的数值幻觉，提出三级可检测性分类，发现领域微调会降低数值约束能力，数值监督反而放大幻觉，指出模板注入是主要机制，并给出评估与部署建议。
中文摘要 AI 辅助
金融大语言模型越来越多地被用于报告和披露文件的摘要生成，其中数值幻觉会带来重大的实际风险。虽然现有研究常将此类幻觉归因于数值推理能力不足，但这一假设尚未在受控微调设置下得到系统验证。本文针对金融摘要生成中的数值幻觉开展了一项高性价比的受控研究，涉及三类模型变体：基础指令微调模型、领域语言适配模型（FT-A）以及数值能力增强的领域模型（FT-A+B+C）。我们提出了一种三级可检测性分类法，将幻觉区分为：显性幻觉（以货币计价的虚构内容）、显性隐性幻觉（符合专业惯例的数字）以及隐性隐性幻觉（无依据的量化表述）。研究结果显示，领域微调会在所有可检测性级别上显著降低数值约束能力。基础模型的幻觉率接近零（5.4%），FT-A的显性幻觉率达82.5%，而FT-A+B+C则达到98%。与直觉相反，数值监督会在所有级别上放大而非缓解幻觉。我们确定模板注入——即无论输入内容如何都插入记忆的标准数值——是微调模型中幻觉的主要机制。这些发现表明，金融摘要生成中的数值幻觉是由领域适配导致的数值约束能力下降驱动的，而非数值推理能力不足。我们建议评估方案应评估所有可检测性级别的幻觉，部署实践应包含基于依据的生成或弃权（不执行）的明确机制。
英文摘要
Financial large language models are increasingly deployed for summarization of reports and disclosures, where numerical hallucination poses significant practical risks. While prior work often attributes such hallucination to insufficient numerical reasoning, this assumption has not been systematically tested under controlled fine-tuning settings. In this paper, we conduct a cost-effective, controlled study of numerical hallucination in financial summarization across three model variants: a base instruction-tuned model, a domain language-adapted model (FT-A), and a numeracy-enhanced domain model (FT-A+B+C). We introduce a three-level detectability taxonomy distinguishing between overt hallucination (currency-denominated fabrication), covert-explicit hallucination (professional-convention numbers), and covert-implicit hallucination (ungrounded quantitative claims). Our results reveal that domain fine-tuning substantially degrades numerical restraint at all detectability levels. While the Base model maintains near-zero hallucination rates (5.4\%), FT-A exhibits 82.5\% overt hallucination and FT-A+B+C reaches 98\%. Contrary to intuition, numeracy supervision amplifies rather than mitigates hallucination across all levels. We identify template injection---the insertion of memorized canonical values regardless of input content---as a primary hallucination mechanism in fine-tuned models. These findings demonstrate that numerical hallucination in financial summarization is driven by the degradation of numerical restraint through domain adaptation, not by insufficient numerical reasoning. We recommend that evaluation protocols assess hallucination across all detectability levels and that deployment practices include explicit mechanisms for grounding-aware generation or abstention.
发表机构
- School of Computer Science, Guangzhou College of Applied Science and Technology（广州应用科技学院计算机学院）
- COFCO Corporation（中粮集团有限公司）
机构由 AI 辅助整理，请以论文原文为准。
