语言大模型 / LLM - arXivDaily 专题

2409.17407 2025-09-23 cs.AI cs.CL 82%

Post-hoc Reward Calibration: A Case Study on Length Bias

Zeyu Huang, Zihan Qiu, Zili Wang, Edoardo M. Ponti, Ivan Titov

机构 * University of Edinburgh（爱丁堡大学）； Alibaba Group（阿里巴巴集团）； INF Technology（INF技术）； University of Amsterdam（阿姆斯特丹大学）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments ICLR 2025

2507.05129 2025-09-19 cs.CL cs.CY cs.LG 82%

SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction

Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, Andrew Lan

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）； Cambium Assessment（Cambium评估）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Published in EMNLP 2025: The 2025 Conference on Empirical Methods in Natural Language Processing

2509.13081 2025-09-17 cs.CL cs.AI 82%

Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPO

Francesco Pappone, Ruggero Marino Lazzaroni, Federico Califano, Niccolò Gentile, Roberto Marras

机构 * AI Sparks（AI火花）； University of Graz（格拉茨大学）； Foyer Group（Foyer集团）； Onepix Academy（Onepix学院）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

2501.06645 2025-07-29 cs.CL cs.AI 82%

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings

Tong Liu, Xiao Yu, Wenxuan Zhou, Jindong Gu, Volker Tresp

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments ACL 2025

2502.18699 2025-07-23 cs.CL cs.LG stat.ME 82%

MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment

Tianze Wang, Dongnan Gui, Yifan Hu, Shuhang Lin, Linjun Zhang

机构 * Department of Statistics, Rutgers University, New Brunswick, United States ； Department of Computer Science, Rutgers University, New Brunswick, United States ； College of Management of Technology, EPFL, Switzerland ； Department of Computer Science, ETH Zurich, Switzerland

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments ICML 2025

2407.06902 2025-07-04 eess.SP cs.AI cs.HC cs.LG 82%

Learning From Crowdsourced Noisy Labels: A Signal Processing Perspective

Shahana Ibrahim, Panagiotis A. Traganitis, Xiao Fu, Georgios B. Giannakis

机构 * Department of Electrical and Computer Engineering, University of Central Florida（中央佛罗里达大学电气与计算机工程系）； School of Electrical Engineering and Computer Science, Oregon State University（俄勒冈州立大学电气工程与计算机科学学院）； Department of Electrical and Computer Engineering, University of Minnesota（明尼苏达大学电气与计算机工程系）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2405.17618 2025-06-24 cs.LG cs.AI 82%

Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales

Ju-Seung Byun, Andrew Perrault

机构 * Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学）； Department of Computer Science（计算机科学系）； Engineering, The Ohio State University（工程系，俄亥俄州立大学）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2506.16502 2025-06-23 cs.CL cs.AI 82%

Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples

Soumya Suvra Ghosal, Vaibhav Singh, Akash Ghosh, Soumyabrata Pal, Subhadip Baidya, Sriparna Saha, Dinesh Manocha

机构 * University of Maryland, College Park（马里兰大学 College Park 分校）； IIT Bombay（印度理工学院班加罗尔分校）； IIT Patna（印度理工学院帕坦分校）； Adobe Research（Adobe 研究院）； IIT Kanpur（印度理工学院坎普尔分校）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；prompting(abstract)

2407.17482 2025-05-27 cs.CY cs.AI cs.CL cs.HC 82%

Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?

Kristian González Barman, Simon Lohse, Henk de Regt

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Journal ref González Barman, K., Lohse, S. & de Regt, H.W. Reinforcement Learning from Human Feedback in LLMs: Whose Culture, Whose Values, Whose Perspectives?. Philos. Technol. 38, 35 (2025)

2505.10775 2025-05-19 cs.CL cs.AI 82%

A Systematic Analysis of Base Model Choice for Reward Modeling

Kian Ahrabian, Pegah Jandaghi, Negar Mokhberian, Sai Praneeth Karimireddy, Jay Pujara

机构 * University of Southern California（南加州大学）； Information Sciences Institute（信息科学研究所）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；post-training(abstract)；RLHF(abstract)

Comments 19 pages, 13 figures, 5 tables

2505.07886 2025-05-14 cs.CL cs.AI 82%

PLHF: Prompt Optimization with Few-Shot Human Feedback

Chun-Pai Yang, Kan Zheng, Shou-De Lin

机构 * ZRT Technology（ZRT技术）； National Taiwan University（国立台湾大学）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2504.02894 2025-04-24 cs.CL cs.AI 82%

OnRL-RAG: Real-Time Personalized Mental Health Dialogue System

Ahsan Bilal, Beiyu Lin

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments It needs more revisions. I am currently working on it with my co-author

2312.00267 2025-03-21 cs.LG cs.AI stat.ML 82%

Sample Efficient Preference Alignment in LLMs via Active Exploration

Viraj Mehta, Syrine Belakaria, Vikramjeet Das, Ojash Neopane, Yijia Dai, Ilija Bogunovic, Barbara Engelhardt, Stefano Ermon, Jeff Schneider, Willie Neiswanger

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2405.15756 2025-02-27 cs.LG cs.AI 82%

Wasserstein Distances, Neuronal Entanglement, and Sparsity

Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir Shavit

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments 10 pages, 9 figures

2406.15599 2025-02-10 cs.LG cs.AI 82%

Pareto-Optimal Learning from Preferences with Hidden Context

Ryan Bahlous-Boldi, Li Ding, Lee Spector, Scott Niekum

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2412.15838 2024-12-31 cs.AI cs.CL 82%

Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback

Jiaming Ji, Jiayi Zhou, Hantao Lou, Boyuan Chen, Donghai Hong, Xuyao Wang, Wenqi Chen, Kaile Wang, Rui Pan, Jiahao Li, Mohan Wang, Josef Dai, Tianyi Qiu, Hua Xu, Dong Li, Weipeng Chen, Jun Song, Bo Zheng, Yaodong Yang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；post-training(abstract)；RLHF(abstract)

2412.10432 2024-12-24 cs.CL cs.AI cs.CR 82%

Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection

Jiaqi Chen, Xiaoye Zhu, Tianyang Liu, Ying Chen, Xinhui Chen, Yiwen Yuan, Chak Tou Leong, Zuchao Li, Tang Long, Lei Zhang, Chenyu Yan, Guanghao Mei, Jie Zhang, Lefei Zhang

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments To appear at AAAI 2025. 14 pages, 6 figure

2412.06827 2024-12-11 cs.LG cs.AI 82%

Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback

Avinash Anand, Kritarth Prasad, Chhavi Kirtani, Ashwin R Nair, Mohit Gupta, Saloni Garg, Anurag Gautam, Snehal Buldeo, Rajiv Ratn Shah

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2405.09223 2024-11-22 cs.CL cs.AI 82%

Word Alignment as Preference for Machine Translation

Qiyu Wu, Masaaki Nagata, Zhongtao Miao, Yoshimasa Tsuruoka

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments EMNLP 2024 Main

2411.02461 2024-11-06 cs.CL cs.AI 82%

Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control

Yuxin Xiao, Chaoqun Wan, Yonggang Zhang, Wenxiao Wang, Binbin Lin, Xiaofei He, Xu Shen, Jieping Ye

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2403.19270 2024-10-08 cs.CL cs.AI 82%

sDPO: Don't Use Your Data All at Once

Dahyun Kim, Yungi Kim, Wonho Song, Hyeonwoo Kim, Yunsu Kim, Sanghoon Kim, Chanjun Park

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2409.19024 2024-10-01 cs.CL cs.AI 82%

Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment

Yan Liu, Xiaoyuan Yi, Xiaokang Chen, Jing Yao, Jingwei Yi, Daoguang Zan, Zheng Liu, Xing Xie, Tsung-Yi Ho

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2406.07657 2024-06-13 cs.LG cs.CL 82%

OPTune: Efficient Online Preference Tuning

Lichang Chen, Jiuhai Chen, Chenxi Liu, John Kirchenbauer, Davit Soselia, Chen Zhu, Tom Goldstein, Tianyi Zhou, Heng Huang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments 16 pages, 7 figures

2312.00886 2024-06-12 stat.ML cs.AI cs.GT cs.LG cs.MA 82%

Nash Learning from Human Feedback

Rémi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Andrea Michi, Marco Selvi, Sertan Girgin, Nikola Momchev, Olivier Bachem, Daniel J. Mankowitz, Doina Precup, Bilal Piot

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2406.02764 2024-06-06 cs.LG cs.AI 82%

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback

Ilgee Hong, Zichong Li, Alexander Bukharin, Yixiao Li, Haoming Jiang, Tianbao Yang, Tuo Zhao

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2404.04626 2024-04-09 cs.CL cs.AI 82%

Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective

Duanyu Feng, Bowen Qin, Chen Huang, Zheng Zhang, Wenqiang Lei

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；SFT(abstract)；preference optimization(abstract)

Comments Draft version

2311.13231 2024-03-26 cs.LG cs.AI cs.CV 82%

Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Qimai Li, Weihan Shen, Xiaolong Zhu, Xiu Li

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments CVPR 2024 accepted; huggingface daily paper

2403.08309 2024-03-15 cs.LG cs.AI 82%

HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback

Ang Li, Qiugen Xiao, Peng Cao, Jian Tang, Yi Yuan, Zijie Zhao, Xiaoyuan Chen, Liang Zhang, Xiangyang Li, Kaitong Yang, Weidong Guo, Yukang Gan, Xu Yu, Daniell Wang, Ying Shan

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments 18 pages, 7 figures

2403.07708 2024-03-15 cs.CL cs.AI 82%

Improving Reinforcement Learning from Human Feedback Using Contrastive Rewards

Wei Shen, Xiaoying Zhang, Yuanshun Yao, Rui Zheng, Hongyi Guo, Yang Liu

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2305.02531 2024-03-01 cs.CL cs.AI 82%

Can LLMs Capture Human Preferences?

Ali Goli, Amandeep Singh

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

1. 后训练与偏好优化 4556 篇

Post-hoc Reward Calibration: A Case Study on Length Bias

SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction

Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPO

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings

MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment

Learning From Crowdsourced Noisy Labels: A Signal Processing Perspective

Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales

Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples

Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?

A Systematic Analysis of Base Model Choice for Reward Modeling

PLHF: Prompt Optimization with Few-Shot Human Feedback

OnRL-RAG: Real-Time Personalized Mental Health Dialogue System

Sample Efficient Preference Alignment in LLMs via Active Exploration

Wasserstein Distances, Neuronal Entanglement, and Sparsity

Pareto-Optimal Learning from Preferences with Hidden Context

Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback

Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection

Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback

Word Alignment as Preference for Machine Translation

Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control

sDPO: Don't Use Your Data All at Once

Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment

OPTune: Efficient Online Preference Tuning

Nash Learning from Human Feedback

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback

Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective

Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback

Improving Reinforcement Learning from Human Feedback Using Contrastive Rewards

Can LLMs Capture Human Preferences?