结合领域专家心智模型的大语言模型增强:通过因果提示工程减少大语言模型幻觉
LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering
AI总结:
本文提出结合专家心智模型(EMM)的因果提示工程框架,通过形式化相关前置流程构建EMM,减少LLM幻觉,在三类任务中验证了方法有效性。
摘要:
当重要决策依赖于未以任何书面形式存在的知识时,大语言模型(LLM)产生幻觉并非源于检索失败,而是源于模型本身未掌握该知识。检索增强生成(RAG)和知识图谱方法存在结构性上限,它们无法提供从未被记录过的内容。本文将这种失败重新概念化为模型发现问题。所提出的因果提示工程框架将领域专家的决策逻辑发现并编码为专家心智模型(Expert Mental Model,EMM),将其作为有原则的输入提供给LLM推理,从而提供任何检索方法都无法提供的内容。该框架基于单调布尔函数和k值函数理论,关键技术贡献在于形式化了三个此前未针对新领域开发的LLM辅助前置流程:因子构建、因子单调化以及保单调层次结构化。这些流程支持四步EMM构建算法,将从数百万种组合中提取的内容减少到人类可处理的、具有形式边界的易处理集合。该框架还首次实现了当专家知识完全隐性时对LLM幻觉的形式化测量。分析显示,对于现有评估方法无法察觉的合理LLM-专家不匹配情况,真阳性的条件错误率可接近100%。该方法在基金提案评估、网络安全设计和临床诊断场景中得到验证。
英文摘要:
When consequential decisions depend on knowledge that exists nowhere in writing, LLMs hallucinate not from retrieval failure but from model absence. RAG and knowledge-graph methods share a structural ceiling. They cannot supply what has never been recorded. This paper reconceptualizes that failure as a model discovery problem. The proposed causal prompt engineering framework discovers and encodes a decision logic of a domain expert as an Expert Mental Model (EMM), providing it as principled input to LLM reasoning supplying what no retrieval method can. The framework is grounded in the theory of monotone Boolean and k-valued functions. A key technical contribution is the formalization of three LLM assisted prerequisite processes previously undeveloped for novel domains: factor formulation, factor monotonization, and monotonicity preserving hierarchical structuring. These enable a four step EMM construction algorithm that reduces elicitation from millions of combinations to a tractable, formally bounded set manageable by humans. The framework also enables, for the first time, formal measurement of LLM hallucination when expert knowledge is entirely tacit. Analysis shows conditional error rates on true positives can approach 100% for plausible LLM-expert mismatches invisible to prior evaluation methods. The approach is validated across grant proposal evaluation, cybersecurity design, and clinical diagnosis.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。