LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood. Existing faithfulness metrics have critical limitations, typically relying on identifying unfaithfulness via adversarial prompting or detecting reasoning errors. These methods overlook the predictive value of explanations. We introduce Normalized Simulatability Gain (NSG), a general and scalable metric based on the idea that a faithful explanation should allow an observer to learn a model's decision-making criteria, and thus better predict its behavior on related inputs. We evaluate 18 frontier proprietary and open-weight models, e.g., Gemini 3, GPT-5.2, and Claude 4.5, on 7,000 counterfactuals from popular datasets covering health, business, and ethics. We find self-explanations substantially improve prediction of model behavior (11-37% NSG). Self-explanations also provide more predictive information than explanations generated by external models, even when those models are stronger. This implies an advantage from self-knowledge that external explanation methods cannot replicate. Our approach also reveals that, across models, 5-15% of self-explanations are highly misleading. Despite their imperfections, we show a positive case for self-explanations: they encode information that helps predict model behavior.
FaithLM: Towards Faithful Explanations for Large Language Models
Rice University（里士大学）；Texas A&M University（德克萨斯农工大学）；Wake Forest University（沃杰森林大学）；New Jersey Institute of Technology（新泽西理工学院）；John Hopkins University（约翰霍普金斯大学）；Google Research（谷歌研究）；Meta Platforms, Inc.（元宇宙平台公司）
When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness
LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Imperial College London（伦敦帝国学院）；Technical University of Munich（慕尼黑工业大学）；MaiNLP lab, CIS, LMU Munich（慕尼黑大学CIS研究所MaiNLP实验室）
Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning
Stratified Precision；Harvard Medical School（哈佛医学院）；Imperial College London（帝国理工学院伦敦分校）；National Health Service（国家健康服务系统）；Ipsen France（Ipsen法国）；University College London（伦敦大学学院）
Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
Quality and Usability Lab, Technische Universität Berlin（柏林技术大学质量与可用性实验室）；University of Copenhagen（哥本哈根大学）；Saarland Informatics Campus（萨尔州信息学校园）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for European Research in Trusted AI (CERTAIN)（可信AI欧洲研究中心）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究院）
