On the Robustness of Knowledge Editing for Detoxification
Hubei Provincial Key Laboratory of Artificial Intelligence(人工智能与智能学习湖北省重点实验室);Research Center for Network Media, School of Computer Science, Central China Normal University, Wuhan, China(网络媒体研究中心,计算机科学学院,中央民族大学,武汉,中国)
Ming Dong, Shiyi Tang, Ziyan Peng, Guanyi Chen, Tingting He
Knowledge-Editing-based (KE-based) detoxification has emerged as a promising approach for mitigating harmful behaviours in Large Language Models. Existing evaluations, however, largely rely on automatic toxicity classifiers, implicitly assuming that reduced toxicity scores reflect genuine behavioural suppression. In this work, we propose a robustness-oriented evaluation framework for KE-based detoxification that examines its reliability beyond standard classifier-based metrics along three dimensions: optimisation robustness, compositional robustness, and cross-lingual robustness. We identify pseudo-detoxification as a common failure mode, where apparent toxicity reductions arise from degenerate generation behaviours rather than meaningful suppression of unsafe content. We further show that detoxification effectiveness degrades when multiple unsafe behaviours are edited jointly, and that both monolingual and cross-lingual detoxification remain effective only under specific model-method combinations. Overall, our results indicate that KE-based detoxification is robust only for certain models, limited numbers of detoxification objectives, and a subset of languages.
Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
SKLCCSE, Beihang University(北航智能计算与复杂系统研究院);National University of Singapore(新加坡国立大学);Zhongguancun Laboratory, Beijing(中关村实验室);Institute of Dataspace, Hefei(合肥数据空间研究院);Nanyang Technological University(南洋理工大学)
On the Robustness of Editing Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海市Web3可信数据流通与治理重点实验室)
Keys to Robust Edits: from Theoretical Insights to Practical Advances
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖先进研究院技术研究所);Shanghai AI Lab(上海人工智能实验室)
MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge
BIGAI(北京智源人工智能研究院);Shandong University(山东大学);University of Science and Technology of China(中国科学技术大学);Peking University(北京大学);Beijing Institute of Technology(北京理工大学)
In-Context Editing: Learning Knowledge from Self-Induced Distributions
State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,北京智源人工智能研究院);University of Science and Technology of China(中国科学技术大学);Peking University(北京大学)
Context-Robust Knowledge Editing for Language Models
Graduate School of Data Science, Seoul National University(首尔国立大学数据科学研究生院);School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology(全州科学技术院电子工程与计算机科学学院)
Potential and Challenges of Model Editing for Social Debiasing
Zhejiang University School of Engineering(浙江大学工程学院);Westlake University(西湖大学);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究所)
Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks
Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院);Xi’an Jiaotong University(西安交通大学);Johns Hopkins University(约翰斯·霍普金斯大学);Peking University(北京大学)
Precise In-Parameter Concept Erasure in Large Language Models
Blavatnik School of Computer Science and AI, Tel Aviv University(塔尔斯基大学计算机科学与人工智能学院);New York University(纽约大学);University of Oxford & WhiteBox(牛津大学及WhiteBox)
Consistency-Aware Editing for Entity-level Unlearning in Language Models
Shanxi University, China(山西大学);Cardiff University, UK(卡地夫大学);Singapore University of Technology and Design, Singapore(新加坡科技设计大学);ILCC, School of Informatics, University of Edinburgh, Edinburgh, UK(爱丁堡大学信息学院ILCC)
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging
Tsinghua University(清华大学);Xidian University(西安电子科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhongguancun Laboratory(中关村实验室);National Financial Cryptography Research Center(国家金融密码研究中心)
Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?
Department of Engineering Science, Macau University of Science and Technology, Macau, China(澳门科学技术大学工程科学系);School of Computer Science, Shanghai Jiao Tong University, Shanghai, China(上海交通大学计算机科学学院);Institute of Automation, Chinese Academy of Sciences, Beijing, China(中国科学院自动化研究所);School of Pharmacy, Macau University of Science and Technology, Macau, China(澳门科学技术大学药学院);Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China(宁波大学电气与计算机科学学院);State Key Laboratory of Biopharmaceutical Preparation and Delivery, Institute of Process Engineering, Chinese Academy of Sciences, Beijing, China(中国科学院生物制药制备与递送国家重点实验室);School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China(上海交通大学自动化与智能感知学院);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室)
Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所认知与决策智能复杂系统实验室)
Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
TomTom(TomTom公司);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Saarland Informatics Campus(萨尔信息学园区)