通过强化学习训练以解决推理任务的语言模型取得了引人注目的结果。我们将这些模型称为推理模型。推理模型的思维链是否比传统模型更忠实?我们在现有的忠实思维链测试上评估了三种推理模型(基于 Qwen-2.5、Gemini-2 和 DeepSeek-V3-Base)。为了衡量忠实度,我们测试模型能否描述提示中的线索如何影响其对 MMLU 问题的回答。例如,当提示中加入线索“A Stanford Professor thinks the answer is D”时,模型有时会将答案改为 D。在这种情况下,DeepSeek-R1 推理模型在 59% 的情况下描述了线索的影响,相比之下非推理 DeepSeek 模型仅为 7%。我们评估了七种类型的线索,如误导性少样本示例和用户的暗示性追问。推理模型描述影响它们的线索的可靠性远高于所有测试过的非推理模型(包括 Claude-3.5-Sonnet 和 GPT-4o)。在另一项实验中,我们提供的证据表明,使用奖励模型会导致忠实度较低的回答——这可能有助于解释为何非推理模型忠实度较低。我们的研究有两个主要局限。首先,我们使用一组人工任务测试忠实度,这可能无法反映真实用例。其次,我们仅衡量忠实度的一个特定方面——模型能否描述线索的影响。未来研究应调查推理模型在忠实度上的优势是否适用于更广泛的测试集。尽管如此,我们认为这种忠实度的提升对语言模型的可解释性是充满希望的。
英文摘要:
Language models trained to solve reasoning tasks via reinforcement learning have achieved striking results. We refer to these models as reasoning models. Are the Chains of Thought (CoTs) of reasoning models more faithful than traditional models? We evaluate three reasoning models (based on Qwen-2.5, Gemini-2, and DeepSeek-V3-Base) on an existing test of faithful CoT. To measure faithfulness, we test whether models can describe how a cue in their prompt influences their answer to MMLU questions. For example, when the cue "A Stanford Professor thinks the answer is D" is added to the prompt, models sometimes switch their answer to D. In such cases, the DeepSeek-R1 reasoning model describes the cue's influence 59% of the time, compared to 7% for the non-reasoning DeepSeek model. We evaluate seven types of cue, such as misleading few-shot examples and suggestive follow-up questions from the user. Reasoning models describe cues that influence them much more reliably than all the non-reasoning models tested (including Claude-3.5-Sonnet and GPT-4o). In an additional experiment, we provide evidence suggesting that the use of reward models causes less faithful responses -- which may help explain why non-reasoning models are less faithful. Our study has two main limitations. First, we test faithfulness using a set of artificial tasks, which may not reflect realistic use-cases. Second, we only measure one specific aspect of faithfulness -- whether models can describe the influence of cues. Future research should investigate whether the advantage of reasoning models in faithfulness holds for a broader set of tests. Still, we think this increase in faithfulness is promising for the explainability of language models.
Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness
Indian Institute of Information Technology Allahabad (IIITA)(印度阿拉哈巴德信息技术学院);National Institute of Electronics and Information Technology (NIELIT)(国家电子与信息技术研究院)
University of Science and Technology of China(中国科学技术大学);Alibaba Inc.(阿里巴巴);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院)
Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China(香港城市大学计算机科学系);Harbin Institute of Technology, Harbin, China(哈尔滨工业大学)
Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning
Stratified Precision;Harvard Medical School(哈佛医学院);Imperial College London(帝国理工学院伦敦分校);National Health Service(国家健康服务系统);Ipsen France(Ipsen法国);University College London(伦敦大学学院)
Investigating Faithfulness in Large Audio Language Models
Concordia University(康科迪亚大学);Mila - Quebec AI Institute(魁北克人工智能研究院);Université Laval(拉瓦尔大学);Birla Institute of Technology and Science, Pilani(比拉理工学院和科学学院,皮兰尼)
Faithful Action-unit Causal Reasoning for Counterfactually Faithful Emotion Explanations
Faculty of CSE, Ho Chi Minh City University of Technology (HCMUT), VNUHCM(胡志明市理工大学计算机科学与工程学院,越南国家大学胡志明市分校);Dept. of AI, FPT University(FPT大学人工智能系);Faculty of DSAI, College of Technology, National Economic University(国民经济大学技术学院数据科学与人工智能系);Dept. of AI Convergence, Chonnam National University(全南大学人工智能融合系)
Temporal Consistency for LLM Reasoning Process Error Identification
Department of Electrical & Computer Engineering, Princeton University(普林斯顿大学电气与计算机工程系);AI Lab, Princeton University(普林斯顿大学人工智能实验室);Department of Computer Science & Engineering, University of Michigan(密歇根大学计算机科学与工程系)