Modern large language models (LLMs) are powerful generators driven by statistical next-token prediction. While effective at producing fluent text, this design biases models toward high-probability continuations rather than exhaustive and faithful answers grounded in source content. As a result, directly applying LLMs lacks systematic mechanisms to ensure both completeness (avoiding omissions) and faithfulness (avoiding unsupported content), which fundamentally conflicts with core AI safety principles. To address this limitation, we present EVE, a structured framework for document-grounded reasoning.
Unlike free-form prompting, EVE constrains generation to a structured, verifiable pipeline that decomposes high-rigor reasoning into extraction, validation, and enumeration. Empirically, this design enables consistent and simultaneous improvements in recall, precision, and F1-score: recall and precision increase by up to 24\% and 29\%, respectively, with a corresponding 31\% gain in F1-score. This effectively breaks the long-standing trade-off between coverage and accuracy typical of single-pass LLM generation, while also mitigating generation truncation caused by length limitations. At the same time, we emphasize that EVE exhibits performance saturation due to the inherent ambiguity of natural language, reflecting fundamental limits of language-based reasoning.
Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
Department of Informatics, University of Oslo(奥斯陆大学信息学院);Integreat – Norwegian Centre for Knowledge-driven Machine Learning(挪威知识驱动机器学习中心);TRUST – The Norwegian Centre for Trustworthy AI(挪威可信人工智能中心)
开卷问答是问答任务的子集,其中系统旨在在给定的文档集合(开卷)和关于主题的常识中找到答案。本文提出了一种解决方案,用于回答来自Amazon Web Services(AWS)技术文档语料库的自然语言问题,其中没有特定于域的标记数据(zero-shot)。这些问题可以有是-否-否答案、简短答案、冗长答案或上述答案的任何组合。该解决方案包括两步架构,其中检索器查找正确的文档,提取器在检索到的文档中查找答案。我们正在为开卷QA引入一个新的测试数据集,该数据集基于AWS技术文档中的真实客户问题。在对几个基于抽取语言模型的信息检索系统和抽取器模型进行实验后,该解决方案尝试在同一遍中找到是-否-否答案和文本答案。该模型在斯坦福问答数据集- SQuAD(Rajpurkaret等人,2016)和自然问题(Kwiatkowski等人,2019)数据集。我们能够实现49%的F1和39%的端到端精确匹配分数(EM),而无需特定于领域的培训。