Large Language Models are commonly judged by their scores on standard benchmarks, yet such scores often overstate real capability since they mask the mix of skills a task actually demands. For example, ARC is assumed to test reasoning, while HellaSwag is designed to evaluate commonsense. However, we lack a systematic way to verify if these benchmarks actually measure these labels. We introduce Benchmark Profiling, a diagnostic framework that decomposes benchmark performance into ten cognitively grounded abilities. The method combines gradient-based importance scoring with targeted parameter ablation to compute an Ability Impact Score (AIS) that quantifies how much each ability contributes to a model's success on a given benchmark. Profiling three instruction-tuned models across ten widely used benchmarks yields four key findings: (i) most benchmarks draw on several abilities rather than one, (ii) datasets with similar labels rely on distinct ability mixtures, (iii) code-generation benchmarks reward broad, multi-skill improvement and thus show only modest gains from narrow domain-specific fine-tuning, and (iv) abilities irrelevant to the task could negatively affect performance. Benchmark Profiling therefore explains why performance gains do not always translate into user-perceived competence and offers a transparent tool for benchmark audit and model interpretability.
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
The University of Hong Kong（香港大学）；Fudan University（复旦大学）；LMU Munich（慕尼黑大学）；Tsinghua University（清华大学）；Technische Universität Darmstadt（达姆施塔特技术大学）；Technische Universität Berlin（柏林技术大学）；Technische Universität Dresden（德累斯顿技术大学）；The Chinese University of Hong Kong（香港中文大学）；University of Pennsylvania（宾夕法尼亚大学）；Nanjing University（南京大学）；University of Manchester（曼彻斯特大学）；Dartmouth College（达特茅斯学院）；University of California Los Angeles（加州大学洛杉矶分校）；University of Michigan（密歇根大学）；Microsoft（微软）；Tencent（腾讯）
A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models
Department of Computer Science and Technology（计算机科学与技术系）；Cancer Research UK（癌症研究英国公司）；Cambridge Institute（剑桥研究所）；University of Cambridge（剑桥大学）；United Kingdom（英国）；DIMES（迪梅斯）；University of Calabria（卡拉布里亚大学）；Italy（意大利）；Department of Computer Automatic and Management Engineering (DIAG)（计算机自动与管理工程系）；Sapienza Università di Roma（罗马大学萨皮恩扎）；Department of Psychiatry（精神病学系）；School of Computing（计算学院）；University of Kent（肯特大学）；Department of Psychology（心理学系）
Dyslexify: A Mechanistic Defense Against Typographic Attacks in CLIP
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；University of Oxford（牛津大学）；Technological University Dublin（都柏林技术大学）；Technische Universität Berlin（柏林技术大学）
LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data
Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg（弗莱堡大学医学中心与医学院医学统计与生物统计研究所）；Department of Dermatology, Medical Faculty and Medical Center, University of Freiburg（弗莱堡大学医学中心与医学院皮肤科）；Prior Labs, University of Freiburg（弗莱堡大学Prior实验室）
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所认知与决策智能复杂系统重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；College of Computer Science, Inner Mongolia University（内蒙古大学计算机学院）
