AI 中文总结
本文针对生成式AI在高风险场景的合理依赖问题，提出含认知谦逊、认知可及性、抗认知不公三条件的规范框架，通过案例分析指出现有标准的不足，为GenAI设计评估提供新方向。
AI 中文摘要
生成式AI系统正越来越多地部署在高风险专业场景中，其输出会影响用户的信念、推理方式以及被视为确定的内容。这为负责任AI提出了核心问题：在何种条件下，对生成式AI输出的依赖在认知上是合理的，而非行为诱导的？现有框架主要关注AI输出是否准确、公平、可解释、安全或被用户信任，这些问题是必要的，且每一项都有助于合理依赖，但它们未将合理依赖明确规定为独立的评估目标——即用户有理由将AI输出作为自身推理输入的条件。本文认为，这需要对认知可信赖性进行阐释：使系统在认知上值得依赖的属性。借鉴哲学中关于可信赖性即能力与受众导向的阐释，本文构建了一个构成性规范框架，包含三个联合必要且不可替代的条件：第一，认知谦逊要求系统表征并传达自身能力的局限；第二，认知可及性要求系统使用户能够在具体情境中检查、质疑和反驳输出；第三，抗认知不公要求系统将用户视为合法的认知主体，避免边缘化其知识与经验。通过法律推理、医疗推理和招聘领域的真实案例分析，本文展示了认知谦逊、认知可及性及抗认知不公的缺失如何产生严重危害，而这些危害无法仅通过标准的准确性、公平性和可用性措施解决。最后，本文概述了围绕认知合理依赖而非仅输出正确性构建的GenAI系统的设计与评估启示。
英文摘要
Generative AI systems are increasingly deployed in high-stakes professional contexts, where their outputs shape what users believe, how they reason, and what they treat as settled. This raises a central question for responsible AI: under what conditions is reliance on generative AI outputs epistemically warranted rather than behaviourally induced? Existing frameworks largely ask whether AI outputs are accurate, fair, explainable, safe, or trusted by users. These questions remain necessary, and each can contribute to warranted reliance. However, they do not directly specify warranted reliance as a distinct evaluative target: the conditions under which users are justified in treating AI outputs as inputs into their own reasoning. We argue that this requires an account of epistemic trustworthiness: what makes a system epistemically worthy of reliance. Drawing on philosophical accounts of trustworthiness as competence and audience-orientation, we develop a constitutive normative framework comprising three jointly necessary and non-fungible conditions. First, epistemic humility requires systems to represent and communicate the limits of their competence. Second, epistemic access requires systems to enable users to inspect, question, and contest outputs in context. Third, resistance to epistemic injustice requires systems to recognise users as legitimate epistemic agents and avoid marginalising their knowledge and experience. Through real-world case analyses in legal reasoning, medical reasoning, and hiring, we show how failures of epistemic humility, epistemic access, and resistance to epistemic injustice can produce consequential harms that standard measures of accuracy, fairness, and usability do not address on their own. We conclude by outlining design and evaluation implications for GenAI systems organised around epistemically warranted reliance rather than output correctness alone.
CommentsAccepted at AAAI/ACM Conference on AI, Ethics, and Society (AIES 2026)
