大模型对齐与安全

Federated Fine-Tuning of Large Language Models: Kahneman-Tversky vs. Direct Preference Optimization

Fernando Spadea, Oshani Seneviratne

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.LG

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, Maosong Sun

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Comments Under review

Process Reward Models for LLM Agents: Practical Framework and Directions

Sanjiban Choudhury

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 7 figures

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

Minsang Kim, Seungjun Baek

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), Findings, Accepted

LLMs Can Teach Themselves to Better Predict the Future

Benjamin Turtel, Danny Franklin, Philipp Schoenegger

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

SimulPL: Aligning Human Preferences in Simultaneous Machine Translation

Donglei Yu, Yang Zhao, Jie Zhu, Yangyifan Xu, Yu Zhou, Chengqing Zong

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025. 23 pages,13 figures,11 tables

PORT: Preference Optimization on Reasoning Traces

Salem Lahlou, Abdalgader Abubaker, Hakim Hacid

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.LG

Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning

Guangxuan Xu, Kai Xu, Shivchander Sudalairaj, Hao Wang, Akash Srivastava

专题命中 偏好对齐 ：safety(abstract)；分类 cs.CL、cs.AI

Atla Selene Mini: A General Purpose Evaluation Model

Andrei Alexandru, Antonia Calvi, Henry Broomfield, Jackson Golden, Kyle Dai, Mathias Leys, Maurice Burger, Max Bartolo, Roman Engeler, Sashank Pisupati, Toby Drane, Young Sun Park

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning

Nirav Diwan, Tolga Ergen, Dongsub Shim, Honglak Lee

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.AI、cs.LG

Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback

Yen-Ting Lin, Di Jin, Tengyu Xu, Tianhao Wu, Sainbayar Sukhbaatar, Chen Zhu, Yun He, Yun-Nung Chen, Jason Weston, Yuandong Tian, Arash Rahnama, Sinong Wang, Hao Ma, Han Fang

专题命中 偏好对齐 ：trustworthy(abstract)；分类 cs.AI、cs.LG

Can LLMs Identify Gaps and Misconceptions in Students' Code Explanations?

Priti Oli, Rabin Banjade, Andrew M. Olney, Vasile Rus

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.AI、cs.CY

Two-Layer Retrieval-Augmented Generation Framework for Low-Resource Medical Question Answering Using Reddit Data: Proof-of-Concept Study

Sudeshna Das, Yao Ge, Yuting Guo, Swati Rajwal, JaMor Hairston, Jeanne Powell, Drew Walker, Snigdha Peddireddy, Sahithi Lakamana, Selen Bozkurt, Matthew Reyna, Reza Sameni, Yunyu Xiao, Sangmi Kim, Rasheeta Chandler, Natalie Hernandez, Danielle Mowery, Rachel Wightman, Jennifer Love, Anthony Spadaro, Jeanmarie Perrone, Abeed Sarker

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Comments Published in JMIR: https://www.jmir.org/2025/1/e66220

HuRef: HUman-REadable Fingerprint for Large Language Models

Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, Zhouhan Lin

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learning

Chirag Nagpal, Subhashini Venugopalan, Jimmy Tobin, Marilyn Ladewig, Katherine Heller, Katrin Tomanek

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ICASSP 2025

CareBot: A Pioneering Full-Process Open-Source Medical Language Model

Lulu Zhao, Weihao Zeng, Xiaofeng Shi, Hua Zhou

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments Accept by AAAI 2025

A Systematic Examination of Preference Learning through the Lens of Instruction-Following

Joongwon Kim, Anirudh Goyal, Aston Zhang, Bo Xiong, Rui Hou, Melanie Kambadur, Dhruv Mahajan, Hannaneh Hajishirzi, Liang Tan

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments 23 pages

Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization

Thomas Savage, Stephen Ma, Abdessalem Boukil, Vishwesh Patel, Ekanath Rangan, Ivan Lopez, Jonathan H Chen

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback

Avinash Anand, Kritarth Prasad, Chhavi Kirtani, Ashwin R Nair, Mohit Gupta, Saloni Garg, Anurag Gautam, Snehal Buldeo, Rajiv Ratn Shah

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.AI、cs.LG

REBEL: Reinforcement Learning via Regressing Relative Rewards

Zhaolin Gao, Jonathan D. Chang, Wenhao Zhan, Owen Oertell, Gokul Swamy, Kianté Brantley, Thorsten Joachims, J. Andrew Bagnell, Jason D. Lee, Wen Sun

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.LG

Comments New experimental results on general chat

Thai Financial Domain Adaptation of THaLLE -- Technical Report

KBTG Labs, Atthakorn Petchsod, Pornchanan Balee, Danupat Khamnuansin, Anuruth Lertpiya, Chanatip Saetia, Tawunrat Chalothorn, Thadpong Pongthawornkamol, Monchai Lertsutthiwong

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models

Hao Yi, Qingyang Li, Yulan Hu, Fuzheng Zhang, Di Zhang, Yong Liu

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.LG

LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning

Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, Wanli Ouyang, Dongzhan Zhou

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Models

Jingyi Chen, Ju-Seung Byun, Micha Elsner, Andrew Perrault

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.AI、cs.LG

Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization

Yuhan Fu, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Xirong Li

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters

Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, Dawn Song

专题命中 偏好对齐 ：safety(abstract)；分类 cs.CL、cs.CY

Comments EMNLP 2024 Main

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

Shengzhi Li, Kittipat Kampa, Rongyu Lin, Bohang Li, Shichao Pei

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Comments We share our latest dataset on https://github.com/findalexli/Abstract2Appendix

Exploring the LLM Journey from Cognition to Expression with Linear Representations

Yuzi Yan, Jialian Li, Yipin Zhang, Dong Yan

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments Published in ICML 2024

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Hyunseok Lee, Jihoon Tack, Jinwoo Shin

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference proceeding for NeurIPS 2024

Little Giants: Synthesizing High-Quality Embedding Data at Scale

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI