认知准确性的Token税：比较RAG与长上下文架构在文档基础生成式AI应用中的表现
The Token Tax of Epistemic Accuracy: Comparing RAG and Long-Context Architectures for Document-Grounded Generative AI Applications
AI总结：
研究比较RAG与长上下文提示两种架构在文档基础生成式AI中的认知准确性与成本权衡，发现长上下文虽准确率更高（73.1% vs 65.4%），但Token成本增加26倍。
摘要：
基于大型语言模型的文档基础助手越来越多地用于高风险、知识密集型工作。然而，其有用性可能取决于生成前证据的分配方式。我们通过比较两种基础架构来研究这一主张：(a) 检索增强生成（RAG），检索少量相关段落；(b) 长上下文提示，将整个文档集合加载到上下文中。我们将这两种架构视为准确-成本前沿上的两种“认知访问”机制。我们用“认知准确性”来捕捉依赖于正确证据的模型正确性。我们假设更广泛的访问（通过长上下文）可以提高认知准确性，但需要付出“Token税”（即由于更大的输入Token消耗而导致成本大幅增加）。我们通过一个制造业安全培训的案例研究来探讨这一框架。使用专家验证的基准，我们评估了跨越三台机器、两个小型语言模型和三种检索/上下文提示方法的972个答案。长上下文提示实现了最高的正确率（73.1%对比语义RAG的65.4%），但每次查询的Token成本是后者的26倍。我们将这一差距解释为更广泛证据访问的Token税。我们仔细讨论了我们的发现对资源受限组织的影响。
英文摘要：
Document-grounded assistants built on large language models are increasingly used in high-stakes, knowledge-intensive work. Their usefulness, however, may depend on how evidence is allocated before generation. We investigate such a claim by comparing two grounding architectures: (a) retrieval-augmented generation (RAG) that retrieves a few relevant passages, and (b) long-context prompting, which loads the whole document collection in context. We view these as two regimes of "epistemic access" on an accuracy--cost frontier. We use "epistemic accuracy" to capture model correctness that depends on having the right evidence. We posit that broader access (via long context) can increase it, but with a "token tax" (i.e., a substantial increase in cost due to larger input token consumption). We probe this framing with a case study in manufacturing safety training. Using an expert-validated benchmark, we evaluate 972 answers across three machines, two small language models, and three retrieval/in-context prompting approaches. Long-context prompting achieved the highest correctness (73.1% vs. 65.4% for semantic RAG), but at 26 times the per-query token cost. We interpret this gap as the token tax of broader evidentiary access. We carefully discuss the implications of our findings for resource-constrained organizations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
