Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
SKLCCSE, Beihang University(北航智能计算与复杂系统研究院);National University of Singapore(新加坡国立大学);Zhongguancun Laboratory, Beijing(中关村实验室);Institute of Dataspace, Hefei(合肥数据空间研究院);Nanyang Technological University(南洋理工大学)
Large Language Models (LLMs) have demonstrated impressive performance across various tasks, yet they remain vulnerable to generating toxic content, necessitating detoxification strategies to ensure safe and responsible deployment. Test-time detoxification methods, which typically introduce static or dynamic interventions into LLM representations, offer a promising solution due to their flexibility and minimal invasiveness. However, current approaches often suffer from imprecise interventions, primarily due to their insufficient exploration of the transition space between toxic and non-toxic outputs. To address this challenge, we propose \textsc{A}utoregressive \textsc{R}eward \textsc{G}uided \textsc{R}epresentation \textsc{E}diting (ARGRE), a novel test-time detoxification framework that explicitly models toxicity transitions within the latent representation space, enabling stable and precise reward-guided editing. ARGRE identifies non-toxic semantic directions and interpolates between toxic and non-toxic representations to reveal fine-grained transition trajectories. These trajectories transform sparse toxicity annotations into dense training signals, enabling the construction of an autoregressive reward model that delivers stable and precise editing guidance. At inference, the reward model guides an adaptive two-step editing process to obtain detoxified representations: it first performs directional steering based on expected reward gaps to shift representations toward non-toxic regions, followed by lightweight gradient-based refinements. Extensive experiments across 8 widely used LLMs show that ARGRE significantly outperforms leading baselines in effectiveness (-62.21% toxicity) and efficiency (-47.58% inference time), while preserving the core capabilities of the original model with minimal degradation. Our code is available at the website.
EtCon: Edit-then-Consolidate for Reliable Knowledge Editing
Wuhan University(武汉大学);Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学)
CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs
Worcester Polytechnic Institute(沃斯特理工学院);Aragon Institute of Engineering Research(阿拉贡工程研究院);Pyrenean Institute of Ecology (IPE-CSIC)(比利牛斯生态研究院(IPE-CSIC))
REDDIT: Forgetting-Resistant Correction of Timestamp Drift in ASR via Replay-Based Distribution Editing
National Taiwan University(台湾大学);Carnegie Mellon University(卡内基梅隆大学);NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)(国立清华大学人工智能研究中心(NTU AI-CoRE))