理性依据传达了什么？角色专业化问答中的消息干预研究
What Do Rationales Communicate? A Message-Intervention Study in Role-Specialized QA
浏览论文内容
中文总结 AI 辅助
本研究通过消息干预诊断，在角色专业化QA中揭示理性依据主要影响验证者的支持判断而非答案准确性，并应作为验证消息机制评估。
中文摘要 AI 辅助
角色专业化的问答流水线日益将理性依据从推理者传递给验证者，但目前尚不清楚这一消息究竟带来了什么：更好的答案、更强的支持评估，还是新的失败面。我们引入了一种消息干预诊断方法，该方法固定证据和候选答案，仅改变跨越推理者到验证者边界传递的理性依据。在400个MuSiQue、HotpotQA和2WikiMultiHopQA示例中，以DeepSeek作为生成器和验证器，忠实的理性依据相比无理性依据几乎没有增加答案准确性，而损坏的理性依据则强烈改变支持判断。在盲验证提示下，无害的改写仅将支持度改变0-2.5%，而损坏的理性依据将支持度改变10-22%；显式的理性依据检查提示将相同模式放大至34-55%。最终答案变动较小（2-30%），且仅有2.9-35.3%的损坏支持翻转与答案变化同时发生。人工审计揭示了其重要性：16/42个有效损坏属于过度信任损坏的情况，而盲审人类拒绝或标记为不清楚9/10个模型接受的被审计损坏理性依据。跨模型和任务边界检查显示了该通道何时活跃、放大、惰性或折叠进任务标签。理性依据共享应作为验证消息机制来评估，而不仅仅是提高答案准确性的途径。
英文摘要
Role-specialized QA pipelines increasingly pass rationales from a reasoner to a verifier, but it is unclear what this message actually buys: better answers, stronger support assessment, or a new failure surface. We introduce a message-intervention diagnostic that fixes the evidence and candidate answer while varying only the rationale passed across the reasoner-to-verifier boundary. On 400 MuSiQue, HotpotQA, and 2WikiMultiHopQA examples with DeepSeek as generator and verifier, faithful rationales add almost no answer accuracy over no rationale, while corrupted rationales strongly alter support judgments. Under a blind verifier prompt, harmless paraphrases shift support by only 0--2.5%, whereas corrupted rationales shift support by 10--22%; an explicit rationale-checking prompt amplifies the same pattern to 34--55%. Final answers move less (2--30%), and only 2.9--35.3% of corrupted support flips co-occur with answer changes. Human audits show why this matters: 16/42 valid corruptions are corruption-overtrust cases, and blind humans reject or mark unclear 9/10 audited corrupted rationales that the model accepts. Cross-model and task-boundary checks show when the channel is active, amplified, inert, or folded into the task label. Rationale sharing should be evaluated as a verification-message mechanism, not merely as a route to higher answer accuracy.
发表机构
- University of Zurich（苏黎世大学）
- Shanghai Jiao Tong University（上海交通大学）
机构由 AI 辅助整理，请以论文原文为准。
