面向金融决策者的幻觉最小化数据到答案框架
Hallucination-minimized Data-to-answer Framework for Financial Decision-makers
AI总结:
提出基于Langchain的框架,将数据表转为文本块,通过意图分类和定制提示实现高置信度问答,减少幻觉,适用于金融决策。
摘要:
大型语言模型(LLMs)已被用于构建多个自动化和个性化问答原型。然而,将这些原型扩展为具有最小化幻觉或虚假响应的稳健产品仍然是一个开放的挑战,尤其是在金融决策等数据表密集的细分领域。在这项工作中,我们提出了一种新颖的基于Langchain的框架,该框架将数据表转换为分层文本数据块,以实现各种可操作的问答。首先,用户查询按意图分类,随后自动检索最相关的数据块,为每个查询生成定制的LLM提示。接下来,定制提示及其响应经过多指标评分,以评估幻觉和响应置信度。所提出的系统通过用户查询意图分类、高级提示、数据扩展能力进行优化,并在各种用户查询响应(包括{什么、哪里、为什么、如何、预测、趋势、异常、例外})中实现了超过90%的置信度分数,这些对于金融决策应用至关重要。所提出的数据到答案框架可扩展到其他分析领域,如销售和薪资,以确保最佳的幻觉控制护栏。
英文摘要:
Large Language Models (LLMs) have been applied to build several automation and personalized question-answering prototypes so far. However, scaling such prototypes to robust products with minimized hallucinations or fake responses still remains an open challenge, especially in niche data-table heavy domains such as financial decision making. In this work, we present a novel Langchain-based framework that transforms data tables into hierarchical textual data chunks to enable a wide variety of actionable question answering. First, the user-queries are classified by intention followed by automated retrieval of the most relevant data chunks to generate customized LLM prompts per query. Next, the custom prompts and their responses undergo multi-metric scoring to assess for hallucinations and response confidence. The proposed system is optimized with user-query intention classification, advanced prompting, data scaling capabilities and it achieves over 90% confidence scores for a variety of user-queries responses ranging from {What, Where, Why, How, predict, trend, anomalies, exceptions} that are crucial for financial decision making applications. The proposed data to answers framework can be extended to other analytical domains such as sales and payroll to ensure optimal hallucination control guardrails.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。