ICLR

Self-Ablating Transformers: More Interpretability, Less Sparsity

Jeremias Ferrao, Luhan Mikaelson, Keenan Pepper, Natalia Perez-Campanero Antolin

Comments Poster Presentation at Building Trust Workshop at ICLR 2025

Conditional Diffusion-Based Retrieval of Atmospheric CO2 from Earth Observing Spectroscopy

William R. Keely, Otto Lamminpää, Steffen Mauceri, Sean M. R. Crowell, Christopher W. O'Dell, Gregory R. McGarragh

机构 * University of Oklahoma（俄克拉荷马大学） ； Jet Propulsion Laboratory（喷气推进实验室） ； California Institute of Technology（加州理工学院） ； LumenUs Scientific（LumenUs科学公司） ； Cooperative Institute for Atmospheric Research（大气研究合作研究所） ； Colorado State University（科罗拉多州立大学） 纠错

Comments Published as a workshop paper in "Tackling Climate Change with Machine Learning", ICLR 2025 Workshop on Tackling Climate Change with Machine Learning. https://www.climatechange.ai/papers/iclr2025/12

Journal ref W. Keely et al. Conditional Diffusion-Based Retrieval of Atmospheric CO2 from Earth Observing Spectroscopy, ICLR 2025 Workshop on Tackling Climate Change with Machine Learning (2025)

EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers

Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, Yujiu Yang

机构 * Tsinghua University（清华大学） ； Microsoft Research（微软研究院） ； Northeastern University（东北大学） 纠错

Comments International Conference on Learning Representations (ICLR) 2024

From GNNs to Trees: Multi-Granular Interpretability for Graph Neural Networks

Jie Yang, Yuwen Wang, Kaixuan Chen, Tongya Zheng, Yihe Zhou, Zhenbang Xiao, Ji Cao, Mingli Song, Shunyu Liu

机构 * Zhejiang University（浙江大学） ； State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室） ； Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江区）区块链与数据安全研究院） ； Big Graph Center, Hangzhou City University（杭州城市大学大数据中心） ； Nanyang Technological University（南洋理工大学） 纠错

Comments Accepted by ICLR 2025

An Empirical Study on Prompt Compression for Large Language Models

Zheng Zhang, Jinyi Li, Yihuai Lan, Xiang Wang, Hao Wang

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； South China University of Technology（华南理工大学） ； University of Science and Technology of China（中国科学技术大学） 纠错

Comments Accepted by Building Trust Workshop at ICLR 2025

Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision

Weicai Yan, Wang Lin, Zirun Guo, Ye Wang, Fangming Feng, Xiaoda Yang, Zehan Wang, Tao Jin

机构 * Zhejiang University（浙江大学） 纠错

Comments Accepted at ICLR 2025

Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation

Sungnyun Kim, Sungwoo Cho, Sangmin Bae, Kangwook Jang, Se-Young Yun

机构 * Kim Jaechul Graduate School of AI, KAIST（金jaechul人工智能研究生院，韩国科学技术院） ； School of Electrical Engineering, KAIST（电气工程学院，韩国科学技术院） 纠错

Comments ICLR 2025; 22 pages, 6 figures, 14 tables

SuoiAI: Building a Dataset for Aquatic Invertebrates in Vietnam

Tue Vo, Lakshay Sharma, Tuan Dinh, Khuong Dinh, Trang Nguyen, Trung Phan, Minh Do, Duong Vu

机构 * Nuoc Solutions ； Microsoft（微软公司） ； University of Oslo（奥斯陆大学） ； Bowdoin College（博德因学院） ； Fulbright University Vietnam（富布赖特大学越南分校） ； Westerdijk Fungal Biodiversity Institute（Westerdijk真菌多样性研究所） 纠错

Comments Published as a workshop paper at "Tackling Climate Change with Machine Learning", ICLR 2025

Elucidating the Preconditioning in Consistency Distillation

Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, Jun Zhu

机构 * Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, THBI Lab（计算机科学与技术系，人工智能研究所，BNRist中心，THBI实验室） ； Tsinghua-Bosch Joint ML Center, Tsinghua University（清华大学-博世联合机器学习中心，清华大学） ； Shengshu Technology（盛舒科技） ； Pazhou Lab (Huangpu), Guangzhou, China（黄埔实验室（广州），中国） 纠错

Comments Accepted at ICLR 2025

FILA: Fine-Grained Vision Language Models

Shiding Zhu, Wenhui Dong, Jun Song, Yingbo Wang, Yanan Guo, Bo Zheng

机构 * Zhejiang University（浙江大学） ； Taobao & Tmall Group of Alibaba（阿里巴巴集团） ； USTC（中国科学技术大学） 纠错

Comments 9 pages, 4 figures, accepted to ICLR 2025 workshop

A Formal Framework for Understanding Length Generalization in Transformers

Xinting Huang, Andy Yang, Satwik Bhattamishra, Yash Sarrof, Andreas Krebs, Hattie Zhou, Preetum Nakkiran, Michael Hahn

Comments 85 pages, 9 figures, 11 tables. Accepted for publication at ICLR 2025

Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling

Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, Qinsheng Zhang

机构 * Department of Computer Science & Technology, Institute for AI, Tsinghua University（1 计算机科学与技术系，人工智能研究院，清华大学） ； NVIDIA（2 英特尔） 纠错

Comments Accepted at ICLR 2025

Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems

Ruochen Jiao, Shaoyuan Xie, Justin Yue, Takami Sato, Lixu Wang, Yixuan Wang, Qi Alfred Chen, Qi Zhu

Comments Accepted paper at ICLR 2025, 31 pages, including main paper, references, and appendix

Diffusion Bridge Implicit Models

Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, Jun Zhu

机构 * Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center（计算机科学与技术系，人工智能研究所，BNRist中心） ； Tsinghua-Bosch Joint ML Center, Tsinghua University（清华大学-博世联合机器学习中心，清华大学） ； Shengshu Technology（盛舒科技） ； Pazhou Lab (Huangpu), Guangzhou, China（黄埔实验室（广州），中国） 纠错

Comments Accepted at ICLR 2025

Emergence of a High-Dimensional Abstraction Phase in Language Transformers

Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Jade Yu, Alessandro Laio, Marco Baroni

机构 * Universitat Pompeu Fabra（庞培法布拉大学） ； Area Science Park（科学公园） ； University of Toronto（多伦多大学） ； SISSA（国际理论物理中心） ； ICREA（伊卡莱研究中心） 纠错

Comments Published as conference paper at ICLR 2025

DDPS: Discrete Diffusion Posterior Sampling for Paths in Layered Graphs

Hao Luan, See-Kiong Ng, Chun Kai Ling

机构 * School of Computing, National University of Singapore（新加坡国立大学计算机学院） ； Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所） 纠错

Comments To appear at Frontiers in Probabilistic Inference: Sampling meets Learning (FPI) workshop at ICLR 2025. https://openreview.net/forum?id=DBdkU0Ikzy

UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation

Huimin Lu, Masaru Isonuma, Junichiro Mori, Ichiro Sakata

Comments Accepted at ICLR 2025 (poster)

MADGEN: Mass-Spec attends to De Novo Molecular generation

Yinkai Wang, Xiaohui Chen, Liping Liu, Soha Hassoun

机构 * Department of Computer Science（计算机科学系） 纠错

Comments ICLR 2025

Benchmarking LLMs' Judgments with No Gold Standard

Shengwei Xu, Yuxuan Lu, Grant Schoenebeck, Yuqing Kong

机构 * School of Information University of Michigan（信息学院密歇根大学） ； School of Computer Science Peking University（计算机科学学院北京大学） 纠错

Comments The Thirteenth International Conference on Learning Representations (ICLR 2025)

QMP: Q-switch Mixture of Policies for Multi-Task Behavior Sharing

Grace Zhang, Ayush Jain, Injune Hwang, Shao-Hua Sun, Joseph J. Lim

机构 * University of Southern California（南加州大学） ； KAIST（韩国科学技术院） ； National Taiwan University（台湾国立大学） 纠错

Comments ICLR 2025

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference

Zhenyu Zhang, Zechun Liu, Yuandong Tian, Harshit Khaitan, Zhangyang Wang, Steven Li

机构 * The University of Texas at Austin（德克萨斯大学奥斯汀分校） ； Meta AI 纠错

Comments ICLR 2025

Atlantes: A system of GPS transformers for global-scale real-time maritime intelligence

Henry Herzog, Joshua Hansen, Yawen Zhang, Patrick Beukema

机构 * Allen Institute for AI (Ai2)（Allen人工智能研究所） 纠错

Comments 8 pages, 10 figures, ICLR CCAI 2025, spotlight talk

PyViT-FUSE: A Foundation Model for Multi-Sensor Earth Observation Data

Manuel Weber, Carly Beneke

机构 * EarthDaily Analytics 纠错

Comments 11 pages, 13 figures, Published at ICLR 2025 - Machine Learning for Remote Sensing (ML4RS) Workshop

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification

Hyunji Jung, Hanseul Cho, Chulhee Yun

Comments 67 pages, 11 figures, accepted to ICLR 2025, Camera-ready version

Support is All You Need for Certified VAE Training

Changming Xu, Debangshu Banerjee, Deepak Vasisht, Gagandeep Singh

机构 * Department of Computer Science University of Illinois Urbana-Champaign（计算机科学系伊利诺伊大学厄巴纳-香槟分校） 纠错

Comments 21 pages, 3 figures, ICLR '25

Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models

Andy Zhou

机构 * Intology AI 纠错

Comments Accepted to ICLR 2025 SCOPE

Toward Understanding In-context vs. In-weight Learning

Bryan Chan, Xinyi Chen, András György, Dale Schuurmans

机构 * University of Alberta（阿尔伯塔大学） ； Google DeepMind（谷歌DeepMind） 纠错

Comments In The Thirteenth International Conference on Learning Representations (ICLR 2025)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models

Michael Noukhovitch, Shengyi Huang, Sophie Xhonneux, Arian Hosseini, Rishabh Agarwal, Aaron Courville

机构 * Mila Quebec AI Institute（魁北克AI研究所） ； Université de Montréal（蒙特利尔大学） ； Allen Institute for AI（人工智能研究院） ； Google Deepmind（谷歌DeepMind） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

Comments accepted at ICLR 2025, code at https://github.com/mnoukhov/async_rlhf, integrated into the open-instruct library https://github.com/allenai/open-instruct

CREAM: Consistency Regularized Self-Rewarding Language Models

Zhaoyang Wang, Weilei He, Zhiyuan Liang, Xuchao Zhang, Chetan Bansal, Ying Wei, Weitong Zhang, Huaxiu Yao

机构 * University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校） ； Nanyang Technological University（南洋理工大学） ； National University of Singapore（国立新加坡大学） ； Microsoft Research（微软研究院） 纠错

Comments To appear at ICLR 2025

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, Boris Hanin

机构 * Princeton Language and Intelligence, Princeton University（普林斯顿语言与智能，普林斯顿大学） 纠错

Comments Accepted to ICLR 2025; Code available at https://github.com/princeton-nlp/unintentional-unalignment