深入探讨大型（视觉）语言模型的偏见与思维链忠实度
A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models
AI总结：
本研究首次全面调查了大型视觉语言模型中基于文本和图像的偏见对思维链忠实度的影响，提出了细粒度评估流水线，并揭示了模型在处理微妙偏见时的不一致推理现象。
摘要：
思维链（CoT）推理增强了大型语言模型的性能，但关于这些推理轨迹是否如实反映了模型的内部过程仍存疑问。我们提出了首项关于大型视觉语言模型（LVLMs）中CoT忠实度的全面研究，调查了基于文本的偏见和此前未探索的基于图像的偏见如何影响推理与偏见表达。我们的工作引入了一种新颖且细粒度的评估流水线，用于对偏见表达模式进行分类，从而能够比以往方法更精确地分析CoT推理。该框架揭示了模型处理和响应不同类型偏见方式的关键差异，为LVLM的CoT忠实度提供了新见解。研究结果表明，与显式的基于文本的偏见相比，微妙的基于图像的偏见很少被表达出来，即使在专门用于推理的模型中也是如此。此外，许多模型表现出一种此前未被识别的现象，我们称之为“不一致”推理——在正确推理后突然改变答案，这可以作为从不忠实的CoT中检测有偏见推理的潜在预警。随后，我们应用相同的评估流水线，重新审视了LLMs在各种隐式线索层面的CoT忠实度。研究结果表明，当前纯语言推理模型在表达未明确陈述的线索方面依然面临困难。
英文摘要：
Chain-of-thought (CoT) reasoning enhances performance of large language models, but questions remain about whether these reasoning traces faithfully reflect the internal processes of the model. We present the first comprehensive study of CoT faithfulness in large vision-language models (LVLMs), investigating how both text-based and previously unexplored image-based biases affect reasoning and bias articulation. Our work introduces a novel, fine-grained evaluation pipeline for categorizing bias articulation patterns, enabling significantly more precise analysis of CoT reasoning than previous methods. This framework reveals critical distinctions in how models process and respond to different types of biases, providing new insights into LVLM CoT faithfulness. Our findings reveal that subtle image-based biases are rarely articulated compared to explicit text-based ones, even in models specialized for reasoning. Additionally, many models exhibit a previously unidentified phenomenon we term ``inconsistent'' reasoning - correctly reasoning before abruptly changing answers, serving as a potential canary for detecting biased reasoning from unfaithful CoTs. We then apply the same evaluation pipeline to revisit CoT faithfulness in LLMs across various levels of implicit cues. Our findings reveal that current language-only reasoning models continue to struggle with articulating cues that are not overtly stated.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
