语言大模型 / LLM - arXivDaily 专题

2511.13765 2025-11-19 cs.LG cs.AI 88%

PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learning

Shengjie Sun, Jiafei Lyu, Runze Liu, Mengbei Yan, Bo Liu, Deheng Ye, Xiu Li

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）

专题命中后训练与偏好优化：LLM(title)；preference optimization(title)；large language model(abstract)；language model(abstract)

2511.10656 2025-11-17 cs.CL cs.AI 88%

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models

Biao Liu, Ning Xu, Junming Yang, Xin Geng

机构 * School of Computer Science and Engineering（计算机科学与工程学院）； Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications（新一代人工智能技术及其交叉应用关键实验室）； Ministry of Education（教育部）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2505.16483 2025-11-13 cs.CL cs.AI 88%

Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning

Shuzheng Si, Haozhe Zhao, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Bofei Gao, Kangyang Luo, Wenhao Li, Yufei Huang, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun

机构 * Shuzheng Si（上海交通大学）； Haozhe Zhao（上海交通大学）； Cheng Gao（上海交通大学）； Yuzhuo Bai（上海交通大学）； Zhitong Wang（上海交通大学）； Bofei Gao（上海交通大学）； Kangyang Luo（上海交通大学）； Wenhao Li（上海交通大学）； Yufei Huang（上海交通大学）； Gang Chen（上海交通大学）； Fanchao Qi（上海交通大学）； Minjia Zhang（上海交通大学）； Baobao Chang（上海交通大学）； Maosong Sun（复旦大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments AAAI 2026 (Oral Presentation)

2509.01308 2025-10-30 cs.AI cs.CL cs.DB 88%

GradeSQL: Test-Time Inference with Outcome Reward Models for Text-to-SQL Generation from Large Language Models

Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2510.01135 2025-10-02 cs.LG cs.CL 88%

Prompt Curriculum Learning for Efficient LLM Post-Training

Zhaolin Gao, Joongwon Kim, Wen Sun, Thorsten Joachims, Sid Wang, Richard Yuanzhe Pang, Liang Tan

机构 * Meta Superintelligence Labs（Meta超智能实验室）； Cornell University（康奈尔大学）； University of Washington（华盛顿大学）

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(title)；language model(abstract)；分类 cs.CL、cs.LG

2509.24261 2025-09-30 cs.AI cs.LG 88%

Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models

Yuhua Jiang, Jiawei Huang, Yufeng Yuan, Xin Mao, Yu Yue, Qianchuan Zhao, Lin Yan

机构 * Tsinghua University（清华大学）； ETH Zurich（苏黎世联邦理工学院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

2508.14460 2025-08-21 cs.LG cs.CL 88%

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization

Shuaijie She, Yu Bao, Yu Lu, Lu Xu, Tao Li, Wenhao Zhu, Shujian Huang, Shanbo Cheng, Lu Lu, Yuxuan Wang

机构 * Nanjing University（南京大学）

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 4 figures,

2508.05613 2025-08-08 cs.CL cs.AI 88%

Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models

Haitao Hong, Yuchen Yan, Xingyu Wu, Guiyang Hou, Wenqi Zhang, Weiming Lu, Yongliang Shen, Jun Xiao

机构 * Zhejiang University（浙江大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Project Page: https://zju-real.github.io/cooper Code: https://github.com/zju-real/cooper

2507.16826 2025-07-24 cs.IR cs.AI cs.CL 88%

A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models

Qikai Wei, Huansheng Ning, Chunlong Han, Jianguo Ding

机构 * School of Computer and Communication Engineering, University of Science and Technology Beijing（计算机与通信工程学院，北京科技大学）； Blekinge Institute of Technology（布莱金厄学院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2505.23848 2025-06-02 cs.CL cs.LG 88%

Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models

Harvey Dam, Jonas Knochelmann, Vinu Joseph, Ganesh Gopalakrishnan

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(title)；large language model(abstract)；分类 cs.CL、cs.LG

2505.16673 2025-05-23 cs.CV cs.AI cs.CL 88%

R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO

Huanjin Yao, Qixiang Yin, Jingyi Zhang, Min Yang, Yibo Wang, Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, Jiaxing Huang

机构 * Nanyang Technological University（南洋理工大学）； ByteDance（字节跳动）； Tsinghua University（清华大学）； Beijing University of Posts and Telecommunications（北京邮电大学）； The University of Sydney（悉尼大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Technical report

2505.07558 2025-05-21 cs.LG cs.CL stat.ML 88%

Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models

Rei Higuchi, Taiji Suzuki

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2505.12476 2025-05-20 cs.CL cs.AI 88%

Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering

Xiao Long, Liansheng Zhuang, Chen Shen, Shaotian Yan, Yifei Li, Shafei Wang

机构 * School of Cyberspace Science and Technology, University of Science and Technology of China (USTC)（信息科学与技术学院，中国科学技术大学）； Peng Cheng Laboratory（鹏城实验室）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2503.16511 2025-03-24 cs.CL cs.AI 88%

Token-Level Uncertainty-Aware Objective for Language Model Post-Training

Tingkai Liu, Ari S. Benjamin, Anthony M. Zador

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

2402.04477 2025-02-28 cs.CL cs.AI 88%

Detecting Mode Collapse in Language Models via Narration

Sil Hamilton

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract,comments)；instruction tuning(abstract)；RLHF(abstract)

Comments To appear in the proceedings of the first Workshop on the Scaling Behavior of Large Language Models (EACL 2024)

Journal ref https://aclanthology.org/2024.scalellm-1.5/

2502.12066 2025-02-18 cs.AI cs.LG cs.SE 88%

CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models

Yifan Zhang, Xue Yang

专题命中后训练与偏好优化：large language model(title)；language model(title)；LLM(abstract)；RLHF(abstract)

2405.15624 2025-01-28 cs.LG cs.AI 88%

Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning

Hao Sun, Mihaela van der Schaar

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

2501.02790 2025-01-07 cs.CL cs.AI 88%

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model

Yueqin Yin, Shentao Yang, Yujia Xie, Ziyi Yang, Yuting Sun, Hany Awadalla, Weizhu Chen, Mingyuan Zhou

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(title,abstract)；分类 cs.CL、cs.AI

2410.03182 2024-11-22 cs.CL cs.AI 88%

Generating bilingual example sentences with large language models as lexicography assistants

Raphael Merx, Ekaterina Vylomova, Kemal Kurniawan

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

2411.11504 2024-11-19 cs.AI cs.CL stat.ML 88%

Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering

Xinyan Guan, Yanjiang Liu, Xinyu Lu, Boxi Cao, Ben He, Xianpei Han, Le Sun, Jie Lou, Bowen Yu, Yaojie Lu, Hongyu Lin

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

2411.00897 2024-11-05 cs.CL cs.AI 88%

Enhancing the Traditional Chinese Medicine Capabilities of Large Language Model through Reinforcement Learning from AI Feedback

Song Yu, Xiaofei Xu, Fangfei Xu, Li Li

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 figures

2410.06554 2024-10-17 cs.CL cs.AI 88%

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models

Yanjun Chen, Dawei Zhu, Yirong Sun, Xinghao Chen, Wei Zhang, Xiaoyu Shen

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 27 figures (including 18 in the appendix), submitted to EMNLP 2024

2410.09362 2024-10-15 cs.LG cs.AI 88%

SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins

Jongwoo Ko, Saket Dingliwal, Bhavana Ganesh, Sailik Sengupta, Sravan Bodapati, Aram Galstyan

专题命中后训练与偏好优化：large language model(title)；language model(title)；RLHF(abstract)；preference optimization(abstract)

2407.13833 2024-08-26 cs.CL cs.AI 88%

Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle

Emman Haider, Daniel Perez-Becker, Thomas Portet, Piyush Madan, Amit Garg, Atabak Ashfaq, David Majercak, Wen Wen, Dongwoo Kim, Ziyi Yang, Jianwen Zhang, Hiteshi Sharma, Blake Bullwinkel, Martin Pouliot, Amanda Minnich, Shiven Chawla, Solianna Herrera, Shahed Warreth, Maggie Engler, Gary Lopez, Nina Chikanov, Raja Sekhar Rao Dheekonda, Bolor-Erdene Jagdagdorj, Roman Lutz, Richard Lundeen, Tori Westerhoff, Pete Bryan, Christian Seifert, Ram Shankar Siva Kumar, Andrew Berkley, Alex Kessler

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

2408.04655 2024-08-13 cs.CL cs.AI 88%

Strong and weak alignment of large language models with human values

Mehdi Khamassi, Marceau Nahon, Raja Chatila

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication in Scientific Reports, special issue on AI aligment

2402.00742 2024-07-22 cs.CL cs.AI 88%

Transforming and Combining Rewards for Aligning Large Language Models

Zihao Wang, Chirag Nagpal, Jonathan Berant, Jacob Eisenstein, Alex D'Amour, Sanmi Koyejo, Victor Veitch

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.CL、cs.AI

2406.11190 2024-06-18 cs.CL cs.AI 88%

Aligning Large Language Models from Self-Reference AI Feedback with one General Principle

Rong Bao, Rui Zheng, Shihan Dou, Xiao Wang, Enyu Zhou, Bo Wang, Qi Zhang, Liang Ding, Dacheng Tao

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 3 figures

2403.08635 2024-03-14 cs.LG cs.AI stat.ML 88%

Human Alignment of Large Language Models through Online Preference Optimisation

Daniele Calandriello, Daniel Guo, Remi Munos, Mark Rowland, Yunhao Tang, Bernardo Avila Pires, Pierre Harvey Richemond, Charline Le Lan, Michal Valko, Tianqi Liu, Rishabh Joshi, Zeyu Zheng, Bilal Piot

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.AI、cs.LG

2305.17760 2024-01-03 cs.CL cs.LG 88%

Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspective

Khanh Nguyen

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(title)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Proceedings of the First Workshop on Theory of Mind in Communicating Agents at (TOM @ ICML 2023)

2310.12304 2023-10-20 stat.ML cs.AI cs.LG 88%

Preference Optimization for Molecular Language Models

Ryan Park, Ryan Theisen, Navriti Sahni, Marcel Patek, Anna Cichońska, Rayees Rahman

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI、cs.LG

1. 后训练与偏好优化 4541 篇

PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learning

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models

Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning

GradeSQL: Test-Time Inference with Outcome Reward Models for Text-to-SQL Generation from Large Language Models

Prompt Curriculum Learning for Efficient LLM Post-Training

Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization

Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models

A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models

Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models

R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO

Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models

Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering

Token-Level Uncertainty-Aware Objective for Language Model Post-Training

Detecting Mode Collapse in Language Models via Narration

CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models

Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model

Generating bilingual example sentences with large language models as lexicography assistants

Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering

Enhancing the Traditional Chinese Medicine Capabilities of Large Language Model through Reinforcement Learning from AI Feedback

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models

SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins

Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle

Strong and weak alignment of large language models with human values

Transforming and Combining Rewards for Aligning Large Language Models

Aligning Large Language Models from Self-Reference AI Feedback with one General Principle

Human Alignment of Large Language Models through Online Preference Optimisation

Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspective

Preference Optimization for Molecular Language Models