We propose probe-based data attribution, a method that traces behavioral changes in post-trained language models to responsible training datapoints. By computing activation-difference vectors for both test prompts and preference pairs and ranking by cosine similarity, we identify datapoints that cause specific behaviors and validate these attributions causally by retraining with modified data. Clustering behavior-datapoint similarity matrices also enables unsupervised discovery of emergent behaviors. Applying this to OLMo 2's production DPO training, we surfaced distractor-triggered compliance: a harmful behavior where the model complies with dangerous requests when benign formatting instructions are appended. Filtering top-ranked datapoints reduces this behavior by 63% while switching their labels achieves 78%. Our method outperforms gradient-based attribution and LLM-judge baselines while being over 10 times cheaper than both. This in-the-wild model organism - emerging from contaminated preference data rather than deliberate injection - provides a realistic benchmark for safety techniques.
Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
Data Science Center ScaDS.AI, Universität Leipzig（ScaDS.AI数据科学中心，莱比锡大学）；Department of Artificial Intelligence, Fraunhofer Heinrich-Hertz-Institute（人工智能系，弗劳恩霍夫 Heinrich-Hertz 研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）
Discovering the Precursors of Traffic Breakdowns Using Spatiotemporal Graph Attribution Networks
Department of Civil Engineering and Engineering Mechanics, Columbia University School of Economics, Minzu University of China（土木工程与工程力学系，哥伦比亚大学经济学院，中国民族大学）；Vehicle and Mobility Systems Department, Argonne National Laboratory（车辆与移动系统部，阿贡国家实验室）；School of Sustainable Engineering and the Built Environment, Arizona State University（可持续工程与环境学院，亚利桑那州立大学）
Beyond Prompt or Skill? Attribution-Guided Optimization of Modular LLM Programs
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)（深圳未来智能网络研究院（深圳FNii））；Sun Yat-sen University（中山大学）
Machine Learning (ML) models are known to be vulnerable to adversarial inputs and researchers have demonstrated that even production systems, such as self-driving cars and ML-as-a-service offerings, are susceptible. These systems represent a target for bad actors. Their disruption can cause real physical and economic harm. When attacks on production ML systems occur, the ability to attribute the attack to the responsible threat group is a critical step in formulating a response and holding the attackers accountable. We pose the following question: can adversarially perturbed inputs be attribut…
The paper proposes summarized attribution-based post-hoc explanations for the detection and identification of bias in data. A global explanation is proposed, and a step-by-step framework on how to detect and test bias is introduced. Since removing unwanted bias is often a complicated and tremendous task, it is automatically inserted, instead. Then, the bias is evaluated with the proposed counterfactual approach. The obtained results are validated on a sample skin lesion dataset. Using the proposed method, a number of possible bias causing artifacts are successfully identified and confirmed in …
University of Colorado, Colorado Springs CO 80918, USA , University of Colorado, Colorado Springs CO 80918, USA
The classification of textual data often yields important information. Most classifiers work in a closed world setting where the classifier is trained on a known corpus, and then it is tested on unseen examples that belong to one of the classes seen during training. Despite the usefulness of this design, often there is a need to classify unseen examples that do not belong to any of the classes on which the classifier was trained. This paper describes the open set scenario where unseen examples from previously unseen classes are handled while testing. This further examines a process of enhanced…
