语言大模型 / LLM

Teaching Models to Balance Resisting and Accepting Persuasion

Elias Stengel-Eskin, Peter Hase, Mohit Bansal

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments NAACL Camera-Ready. Code: https://github.com/esteng/persuasion_balanced_training

LLMs Can Teach Themselves to Better Predict the Future

Benjamin Turtel, Danny Franklin, Philipp Schoenegger

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Jackpot! Alignment as a Maximal Lottery

Roberto-Rafael Maura-Rivero, Marc Lanctot, Francesco Visin, Kate Larson

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Feasible Learning

Juan Ramirez, Ignacio Hounie, Juan Elenter, Jose Gallego-Posada, Meraj Hashemizadeh, Alejandro Ribeiro, Simon Lacoste-Julien

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

Comments Published at AISTATS 2025. Code available at https://github.com/juan43ramirez/feasible-learning

Clone-Robust AI Alignment

Ariel D. Procaccia, Benjamin Schiffer, Shirley Zhang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information

Rasul Tutnov, Antoine Grosnit, Haitham Bou-Ammar

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

COPR: Continual Human Preference Learning via Optimal Policy Regularization

Han Zhang, Lin Gui, Yu Lei, Yuanzhao Zhai, Yehong Zhang, Yulan He, Hui Wang, Yue Yu, Kam-Fai Wong, Bin Liang, Ruifeng Xu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments This is a duplicate submission to arXiv:2310.15694, and we believe that this submission has affected the citation of our original paper arXiv:2310.15694

SafeWorld: Geo-Diverse Safety Alignment

Da Yin, Haoyi Qiu, Kung-Hsiang Huang, Kai-Wei Chang, Nanyun Peng

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2024

Unsupervised Human Preference Learning

Sumuk Shashidhar, Abhinav Chinta, Vaibhav Sahai, Dilek Hakkani-Tür

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Main Conference

Journal ref EMNLP 2024

Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment

Ran Tian, Yilin Wu, Chenfeng Xu, Masayoshi Tomizuka, Jitendra Malik, Andrea Bajcsy

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Submitted to IJRR, this paper is an extended journal version of the conference paper arXiv:2310.07932 with new results and discussion. arXiv admin note: substantial text overlap with arXiv:2310.07932

Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

Yu Bai, Yukai Miao, Li Chen, Dawei Wang, Dan Li, Yanyu Ren, Hongtao Xie, Ce Yang, Xuhui Cai

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs

Rui Yang, Ruomeng Ding, Yong Lin, Huan Zhang, Tong Zhang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

Personalized Adaptation via In-Context Preference Learning

Allison Lau, Younwoo Choi, Vahid Balazadeh, Keertana Chidambaram, Vasilis Syrgkanis, Rahul G. Krishnan

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

PopAlign: Diversifying Contrasting Patterns for a More Comprehensive Alignment

Zekun Moore Wang, Shawn Wang, Kang Zhu, Jiaheng Liu, Ke Xu, Jie Fu, Wangchunshu Zhou, Wenhao Huang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments 28 pages

Anchored Alignment for Self-Explanations Enhancement

Luis Felipe Villa-Arenas, Ata Nizamoglu, Qianli Wang, Sebastian Möller, Vera Schmitt

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation

Xiaoying Zhang, Jean-Francois Ton, Wei Shen, Hongning Wang, Yang Liu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Orchestrating LLMs with Different Personalizations

Jin Peng Zhou, Katie Z Luo, Jingwen Gu, Jason Yuan, Kilian Q. Weinberger, Wen Sun

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

WARP: On the Benefits of Weight Averaged Rewarded Policies

Alexandre Ramé, Johan Ferret, Nino Vieillard, Robert Dadashi, Léonard Hussenot, Pierre-Louis Cedoz, Pier Giuseppe Sessa, Sertan Girgin, Arthur Douillard, Olivier Bachem

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments 11 main pages (34 pages with Appendix)

On the Transformations across Reward Model, Parameter Update, and In-Context Prompt

Deng Cai, Huayang Li, Tingchen Fu, Siheng Li, Weiwen Xu, Shuaiyi Li, Bowen Cao, Zhisong Zhang, Xinting Huang, Leyang Cui, Yan Wang, Lemao Liu, Taro Watanabe, Shuming Shi

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Generalizing Reward Modeling for Out-of-Distribution Preference Learning

Chen Jia

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments 31 pages

LIRE: listwise reward enhancement for preference alignment

Mingye Zhu, Yi Liu, Lei Zhang, Junbo Guo, Zhendong Mao

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted by ACL 2024 Findings

Aligning LLMs through Multi-perspective User Preference Ranking-based Feedback for Programming Question Answering

Hongyu Yang, Liyang He, Min Hou, Shuanghong Shen, Rui Li, Jiahui Hou, Jianhui Ma, Junda Zhao

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Prompt Optimization with Human Feedback

Xiaoqiang Lin, Zhongxiang Dai, Arun Verma, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Preprint, 18 pages

Adversarial DPO: Harnessing Harmful Data for Reducing Toxicity with Minimal Impact on Coherence and Evasiveness in Dialogue Agents

San Kim, Gary Geunbae Lee

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 7 figures, accepted to NAACL findings 2024

Understanding the performance gap between online and offline alignment algorithms

Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, Will Dabney

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Contrastive Preference Learning: Learning from Human Feedback without RL

Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W. Bradley Knox, Dorsa Sadigh

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2024. Code released at https://github.com/jhejna/cpl

Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oracles

Zhiwei Tang, Dmitry Rybin, Tsung-Hui Chang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2024

Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation

Zhenrui Yue, Huimin Zeng, Yimeng Lu, Lanyu Shang, Yang Zhang, Dong Wang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2024

Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections

Lihan Zha, Yuchen Cui, Li-Heng Lin, Minae Kwon, Montserrat Gonzalez Arenas, Andy Zeng, Fei Xia, Dorsa Sadigh

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 4 figures, videos and code links on website https://sites.google.com/stanford.edu/droc

Investigating Bias Representations in Llama 2 Chat via Activation Steering

Dawn Lu, Nina Rimsky

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI