LLM推荐的认知保证：当真实值不可用时表征依赖的基础
Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailable
浏览论文内容
中文总结 AI 辅助
针对LLM推荐缺乏可靠依赖依据的问题，引入认知保证构造并通过四级依赖证书实现，经验证其能有效表征LLM推荐的依赖基础，与置信度、决策难度无关。
中文摘要 AI 辅助
大型语言模型（LLM）正越来越多地用于支持组织决策，但用户往往缺乏评估是否依赖某一特定推荐的原则性基础。现有方法通常评估模型的广泛属性，如可靠性、不确定性或鲁棒性，或聚焦于用户信任，而非依赖单个推荐的潜在基础。我们借鉴认识论的理论基础，引入认知保证（epistemic warrant）这一决策层面的构造，用以表征模型偏好的稳定性及该偏好适用的范围。我们通过针对成对推荐的四级依赖证书来实现这一构造，区分不稳定、依赖上下文、局部支持和广泛支持的推荐。我们采用当代方法论验证该构造：已知组测试成功恢复了专家预先指定的保证排序，且更强的保证与众包工作者的独立共识系统地一致。此外，我们证明认知保证提供的信息与口头表达的置信度不同，且无法用决策难度轻易解释。最终，该框架提供了一种理论上有依据、可实施的方法，用于在客观真实值不可用时表征单个LLM推荐的保证。
英文摘要
Large language models are increasingly used to support organizational decisions, yet users often lack a principled basis for assessing whether to rely on a specific recommendation. Existing approaches typically evaluate broad model properties, such as reliability, uncertainty, or robustness, or focus on user trust, rather than the underlying basis for relying on an individual recommendation. Adapting theoretical foundations from epistemology, we introduce epistemic warrant, a decision-level construct that characterizes the stability of a model's preference and the scope over which that preference holds. We operationalize this construct through a four-tier reliance certificate for pairwise recommendations, distinguishing among unstable, context-dependent, locally supported, and broadly supported recommendations. We validate the construct using contemporary methodologies: known-groups tests successfully recover expert-prespecified warrant orderings, and stronger warrants systematically align with independent consensus from crowd workers. Furthermore, we demonstrate that epistemic warrant provides information distinct from verbalized confidence and is not readily explained by decision difficulty. Ultimately, this framework offers a theoretically grounded, implementable approach for characterizing the warrant of individual LLM recommendations when objective ground truth is unavailable.
发表机构
- University of South Florida（南佛罗里达大学）
- Muma College of Business（马玛商学院）
- New York University（纽约大学）
机构由 AI 辅助整理，请以论文原文为准。
