语言大模型 / LLM

ResNets Are Deeper Than You Think

Christian H. X. Ali Mehmeti-Göpel, Michael Wand

机构 * Department of Computer Science Johannes-Gutenberg University Mainz, Germany（计算机科学系格鲁特伯爵大学马因茨德国） 纠错

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2025 Submission

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment

Jay Hyeon Cho, JunHyeok Oh, Myunsoo Kim, Byung-Jun Lee

机构 * Korea University（韩国大学） 纠错

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.CL、cs.AI

AssistanceZero: Scalably Solving Assistance Games

Cassidy Laidlaw, Eli Bronstein, Timothy Guo, Dylan Feng, Lukas Berglund, Justin Svegliato, Stuart Russell, Anca Dragan

机构 * ucb（加州大学伯克利分校） 纠错

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments Presented at ICML 2025

Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences

Weijian Luo

机构 * Peking University（北京大学） 纠错

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments Revision: The paper was accepted by Transactions of Machine Learning Research (TMLR)

Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns

Xiang Li, Haiyang Yu, Xinghua Zhang, Ziyang Huang, Shizhu He, Kang Liu, Jun Zhao, Fei Huang, Yongbin Li

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Tongyi Lab, Alibaba Group（阿里云实验室） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；分类 cs.CL、cs.AI

A Practical Analysis of Human Alignment with *PO

Kian Ahrabian, Xihui Lin, Barun Patra, Vishrav Chaudhary, Alon Benhaim, Jay Pujara, Xia Song

机构 * University of Southern California, Information Sciences Institute（南加州大学信息科学研究所） ； Microsoft（微软公司） 纠错

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2025 findings papers. 9 pages, 7 figures, 3 tables

Enhancing Variational Autoencoders with Smooth Robust Latent Encoding

Hyomin Lee, Minseon Kim, Sangwon Jang, Jongheon Jeong, Sung Ju Hwang

机构 * Korea University（韩国大学） ； Microsoft（微软） ； KAIST（韩国科学技术院） 纠错

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Comments Under review

Learning a Canonical Basis of Human Preferences from Binary Ratings

Kailas Vodrahalli, Wei Wei, James Zou

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 11 figures

Latent Embedding Adaptation for Human Preference Alignment in Diffusion Planners

Wen Zheng Terence Ng, Jianda Chen, Yuan Xu, Tianwei Zhang

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments 8 pages

What is the Alignment Objective of GRPO?

Milan Vojnovic, Se-Young Yun

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

PAD: Personalized Alignment of LLMs at Decoding-Time

Ruizhe Chen, Xiaotian Zhang, Meng Luo, Wenhao Chai, Zuozhu Liu

专题命中 后训练与偏好优化 ：LLM(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025

Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking

Cassidy Laidlaw, Shivam Singhal, Anca Dragan

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments Spotlight at ICLR 2025

Behavior Preference Regression for Offline Reinforcement Learning

Padmanaba Srinivasan, William Knottenbelt

专题命中 后训练与偏好优化 ：language model(abstract)；分类 cs.AI、cs.LG

Comments Conference paper at AAAI 25

Reward Learning from Multiple Feedback Types

Yannick Metz, András Geiszl, Raphaël Baur, Mennatallah El-Assady

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2025

Is Free Self-Alignment Possible?

Dyah Adila, Changho Shin, Yijing Zhang, Frederic Sala

专题命中 后训练与偏好优化 ：language model(abstract)；分类 cs.CL、cs.LG

Process Reward Model with Q-Value Rankings

Wendi Li, Yixuan Li

专题命中 后训练与偏好优化 ：language model(abstract)；分类 cs.CL、cs.AI

Self-Regulation and Requesting Interventions

So Yeon Min, Yue Wu, Jimin Sun, Max Kaufmann, Fahim Tajwar, Yonatan Bisk, Ruslan Salakhutdinov

专题命中 后训练与偏好优化 ：LLM(abstract)；分类 cs.CL、cs.LG

SimulPL: Aligning Human Preferences in Simultaneous Machine Translation

Donglei Yu, Yang Zhao, Jie Zhu, Yangyifan Xu, Yu Zhou, Chengqing Zong

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025. 23 pages,13 figures,11 tables

Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking

Jie Ren, Yuhang Zhang, Dongrui Liu, Xiaopeng Zhang, Qi Tian

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.AI、cs.LG

Cross-lingual Transfer of Reward Models in Multilingual Alignment

Jiwoo Hong, Noah Lee, Rodrigo Martínez-Castaño, César Rodríguez, James Thorne

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2025

A novel Reservoir Architecture for Periodic Time Series Prediction

Zhongju Yuan, Geraint Wiggins, Dick Botteldooren

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Technical Report for the Forgotten-by-Design Project: Targeted Obfuscation for Machine Learning

Rickard Brännvall, Laurynas Adomaitis, Olof Görnerup, Anass Sedrati

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 4 figures

Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents

Jonathan Keane, Sam Keyser, Jeremy Kedziora

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Federated Unlearning with Gradient Descent and Conflict Mitigation

Zibin Pan, Zhichao Wang, Chi Li, Kaiyan Zheng, Boqi Wang, Xiaoying Tang, Junhua Zhao

专题命中 后训练与偏好优化 ：post-training(abstract)；分类 cs.AI、cs.LG

Comments To be published in the Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI-25)

VideoDPO: Omni-Preference Alignment for Video Diffusion Generation

Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei, Yingqing He, Renjie Pi, Qifeng Chen

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.AI、cs.LG

Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective

Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, Xipeng Qiu

专题命中 后训练与偏好优化 ：LLM(abstract)；分类 cs.AI、cs.LG

Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking

Paria Rashidinejad, Yuandong Tian

专题命中 后训练与偏好优化 ：preference optimization(abstract)；分类 cs.AI、cs.LG

Comments 46 pages, 3 figures

Contextual Bilevel Reinforcement Learning for Incentive Alignment

Vinzenz Thoma, Barna Pasztor, Andreas Krause, Giorgia Ramponi, Yifan Hu

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments 60 pages, 21 Figures

When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback

Leon Lang, Davis Foote, Stuart Russell, Anca Dragan, Erik Jenner, Scott Emmons

专题命中 后训练与偏好优化 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments Advances in Neural Information Processing Systems 37 (NeurIPS 2024)

Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text

Reuben Luera, Ryan Rossi, Franck Dernoncourt, Alexa Siu, Sungchul Kim, Tong Yu, Ruiyi Zhang, Xiang Chen, Nedim Lipka, Zhehao Zhang, Seon Gyeom Kim, Tak Yeon Lee

专题命中 后训练与偏好优化 ：LLM(abstract)；分类 cs.AI、cs.LG