语言大模型 / LLM - arXivDaily 专题

2510.21090 2025-10-27 cs.CL cs.AI cs.LG 90%

Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only

Qingru Zhang, Liang Qiu, Ilgee Hong, Zhenghao Xu, Tianyi Liu, Shiyang Li, Rongzhi Zhang, Zheng Li, Lihong Li, Bing Yin, Chao Zhang, Jianshu Chen, Haoming Jiang, Tuo Zhao

机构 * Georgia Institute of Technology（佐治亚理工学院）； Amazon（亚马逊）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by COLM 2025

2410.23022 2025-10-27 cs.LG cs.AI cs.CL cs.RO 90%

Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback

Qinqing Zheng, Mikael Henaff, Amy Zhang, Aditya Grover, Brandon Amos

机构 * \dagger（机构1）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments RLC 2025

2410.12854 2025-10-27 cs.CL cs.AI 90%

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

Weibin Liao, Xu Chu, Yasha Wang

机构 * School of Computer Science, Peking University（北京大学计算机科学系）； Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）； National Research and Engineering Center of Software Engineering, Peking University（北京大学软件工程研究中心）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICLR 2025

2506.18631 2025-10-27 cs.LG cs.AI cs.CL 87%

ReDit: Reward Dithering for Improved LLM Policy Optimization

Chenxing Wei, Jiarui Yu, Ying Tiffany He, Hande Dong, Yao Shu, Fei Yu

机构 * Guangdong Lab of AI and Digital Economy (SZ), China（广东人工智能与数字经济实验室（深圳））； College of Computer Science and Software Engineering, Shenzhen University, China（深圳大学计算机科学与软件工程学院）； Hong Kong University of Science and Technology (Guangzhou), China（香港科技大学（广州））； Tencent, Shenzhen, China（腾讯（深圳））； School of Information Technology, Carleton University, Canada（卡尔顿大学信息技术学院）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 34 pages, 19 figures

2509.21798 2025-10-27 cs.CL cs.AI 84%

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment

Hongbin Zhang, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang

机构 * Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（计算与智能研究所，哈尔滨工业大学，深圳，中国）； Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳，中国）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Under review;Work in progress;

2504.06020 2025-10-27 cs.AI cs.CL cs.LG 82%

Information-Theoretic Reward Decomposition for Generalizable RLHF

Liyuan Mao, Haoran Xu, Amy Zhang, Weinan Zhang, Chenjia Bai

机构 * Shanghai Jiao Tong University（上海交通大学）； UT Austin（得克萨斯大学奥斯汀分校）； Institute of Artificial Intelligence, China Telecom（中国电信人工智能研究院）

专题命中后训练与偏好优化：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work done during internships at Institute of Artificial Intelligence (TeleAI), China Telecom

2505.11475 2025-10-27 cs.CL cs.AI cs.LG 80%

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages

Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Hoo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, Oleksii Kuchaiev

机构 * NVIDIA

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2025 Datasets and Benchmarks Track Camera Ready, 46 pages, 2 figures

2505.11080 2025-10-27 cs.CL cs.AI cs.LG 75%

BLEUBERI: BLEU is a surprisingly effective reward for instruction following

Yapei Chang, Yekyung Kim, Michael Krumdick, Amir Zadeh, Chuan Li, Chris Tanner, Mohit Iyyer

机构 * University of Maryland, College Park（马里兰大学学院公园分校）

专题命中后训练与偏好优化：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments neurips cam-ready

2505.17859 2025-10-27 cs.LG cs.AI stat.ML 73%

Scalable Valuation of Human Feedback through Provably Robust Model Alignment

Masahiro Fujisawa, Masaki Adachi, Michael A. Osborne

机构 * The University of Osaka（大阪大学）； Lattice Lab, Toyota Motor Corporation（丰田公司Lattice实验室）； Machine Learning Research Group, University of Oxford（牛津大学机器学习研究组）； RIKEN AIP（理化学研究所AIP）

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Accepted by the 39th Conference on Neural Information Processing Systems (NeurIPS2025), 49 pages, 7 figures

2510.21223 2025-10-27 cs.LG 70%

Model Merging with Functional Dual Anchors

Kexuan Shi, Yandong Wen, Weiyang Liu

机构 * The Chinese University of Hong Kong（香港中文大学）； Westlake University（西湖大学）

专题命中后训练与偏好优化：foundation model(abstract)；post-training(abstract)；分类 cs.LG

Comments Technical report (23 pages, 15 figures, project page: https://spherelab.ai/fda/)

1. 后训练与偏好优化 10 篇