语言大模型 / LLM - arXivDaily 专题

2601.18306 2026-01-27 cs.CL cs.AI 86%

Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM

超越英语的校准：为更好的量化多语言大语言模型的语言多样性

Everlyn Asiko Chimoto, Mostafa Elhoushi, Bruce A. Bassett

机构 * Lelapa AI ； Cerebras Systems, Inc ； University of the Witwatersrand（乌得勒支大学）； University of Cape Town（开普敦大学）； South African Astronomical Observatory（南非天文台）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

AI总结本文通过多语言校准集提升多语言大语言模型的量化性能，发现非英语和多语言混合校准显著降低困惑度，强调语言对齐的重要性。

Comments Accepted to EACL 2026 Main Conference

2601.14686 2026-01-22 cs.AI cs.LG 86%

IB-GRPO: Aligning LLM-based Learning Path Recommendation with Educational Objectives via Indicator-Based Group Relative Policy Optimization

IB-GRPO: 通过基于指标的群体相对策略优化对基于大语言模型的学习路径推荐进行对齐

Shuai Wang, Yaoming Yang, Bingdong Li, Hao Hao, Aimin Zhou

机构 * East China Normal University（东华大学）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结 IB-GRPO通过基于指标的群体相对策略优化，解决大语言模型在学习路径推荐中的多目标对齐问题，提升学习效果与教学目标的一致性。

2601.11379 2026-01-19 cs.CL cs.AI cs.CY cs.SI 86%

Evaluating LLM Behavior in Hiring: Implicit Weights, Fairness Across Groups, and Alignment with Human Preferences

评估LLM在招聘中的行为：隐含权重、群体公平性及与人类偏好的一致性

Morgane Hoffmann, Emma Jouffroy, Warren Jouanneau, Marc Palyart, Charles Pebereau

机构 * Malt

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本文提出评估LLM招聘决策的框架，发现模型重视核心生产力信号，但对某些特征的解释超出显性匹配价值，且不同群体间权重存在差异。

2601.07593 2026-01-13 cs.AR cs.CL cs.LG 86%

GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation

GRPO与状态突变：改进基于LLM的硬件测试计划生成

Dimple Vijay Kochar, Nathaniel Pinckney, Guan-Ting Liu, Chia-Tung Ho, Chenhui Deng, Haoxing Ren, Brucek Khailany

机构 * Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA（麻省理工学院电子工程与计算机科学系）； NVIDIA Research, Austin, TX（NVIDIA研究部）； NVIDIA Research, Taiwan（NVIDIA台湾研究部）； NVIDIA Research, Santa Clara, CA（NVIDIA圣克拉拉研究部）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

AI总结 GRPO-SMu通过改进LLM训练方法，显著提升硬件验证中测试计划生成的准确率和突变检测率。

2601.07199 2026-01-13 cs.LG cs.AI 86%

Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization

正向与反向：在直接偏好优化中比较推理目标

Murtaza Nikzad, Raghuram Ramanujan

机构 * Department of Math and Computer Science（数学与计算机科学系）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结本文通过直接偏好优化比较正向与反向推理目标，发现正向训练提升准确性，反向训练降低误报率，两者互补提升模型推理可靠性。

2409.00162 2025-12-25 cs.CL cs.AI 86%

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback

序列到序列奖励建模：通过语言反馈改进RLHF

Jiayi Zhou, Jiaming Ji, Juntao Dai, Dong Li, Yaodong Yang

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结通过序列到序列奖励建模方法改进RLHF，提升LLMs对齐人类意图的性能。

Comments 7 pages

2512.13240 2025-12-16 cs.AI cs.LG 86%

Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection

反射偏好优化（RPO）：通过提示引导的反思增强策略对齐

Zihui Zhao, Zechang Li

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）； Alibaba Group（阿里巴巴集团）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

AI总结 RPO通过提示引导的反思增强策略对齐，减少幻觉并提升多模态基准性能。

2512.10040 2025-12-12 cs.LG cs.AI stat.ML 86%

Intelligently Weighting Multiple Reference Models for Direct Preference Optimization of LLMs

智能加权多个参考模型以直接优化大语言模型的偏好

Skyler Wu, Aymen Echarghaoui

机构 * Stanford University（斯坦福大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结本文提出四种新的加权策略以优化大语言模型的偏好，发现单参考DPO在多数情况下优于多参考方法。

Comments Working paper. 13 pages, 4 figures

2512.07611 2025-12-09 cs.AI cs.LG 86%

Comparative Analysis and Parametric Tuning of PPO, GRPO, and DAPO for LLM Reasoning Enhancement

PPO、GRPO和DAPO在LLM推理增强中的比较分析与参数调优

Yongsheng Lian

机构 * Mechanical Engineering Department University of Louisville（路易斯维尔大学机械工程系）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结本文通过比较PPO、GRPO和DAPO在LLM推理增强中的表现，发现GRPO和DAPO在参数调优下具有更稳定的训练动态和更高的准确性，而DAPO禁用动态采样时表现最佳。

2505.16690 2025-11-26 cs.LG cs.AI 86%

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

你的预训练大语言模型实际上是一个未监督的置信度校准器

Beier Luo, Shuoyuan Wang, Sharon Li, Hongxin Wei

机构 * Department of Statistics and Data Science, Southern University of Science and Technology（统计与数据科学系，南方科技大学）； Department of Computer Sciences, University of Wisconsin-Madison（计算机科学系，威斯康星大学麦迪逊分校）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

AI总结本文提出DACA方法，通过无监督方式优化后训练模型的置信度校准，减少不一致预测带来的过度自信问题，提升模型可靠性。

Comments NeurIPS 2025

2511.12573 2025-11-18 cs.CL cs.AI 86%

Mitigating Length Bias in RLHF through a Causal Lens

Hyeonji Kim, Sujeong Oh, Sanghack Lee

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2511.08325 2025-11-12 cs.CL cs.IR cs.LG 86%

AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress

Zhiheng Xi, Chenyang Liao, Guanyu Li, Yajie Yang, Wenxiang Chen, Zhihao Zhang, Binghai Wang, Senjie Jin, Yuhao Zhou, Jian Guan, Wei Wu, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

机构 * Fudan University（复旦大学）； Ant Group（蚂蚁集团）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2505.23349 2025-11-12 cs.LG cs.AI 86%

Towards Reward Fairness in RLHF: From a Resource Allocation Perspective

Sheng Ouyang, Yulan Hu, Ge Chen, Qingyang Li, Fuzheng Zhang, Yong Liu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）； Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大型模型与智能治理研究重点实验室）； Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE（下一代智能搜索与推荐工程技术研究中心，教育部）； Kuaishou Technology（快手科技）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to ACL 2025

2511.07477 2025-11-12 cs.CY cs.AI cs.CL 86%

The Polite Liar: Epistemic Pathology in Language Models

Bentley DeVilling

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 2 tables, Preprint - under review at AI & Society

2505.13487 2025-11-03 cs.CL cs.AI 86%

Detecting Prefix Bias in LLM-based Reward Models

Ashwin Kumar, Yuzi He, Aram H. Markosyan, Bobbie Chern, Imanol Arrieta-Ibarra

机构 * Washington University in St Louis（华盛顿大学圣路易斯分校）； Meta Platforms, Inc.（Meta平台公司）

专题命中后训练与偏好优化：LLM(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

2510.19178 2025-10-28 cs.LG cs.AI 86%

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs

Runzhe Wu, Ankur Samanta, Ayush Jain, Scott Fujimoto, Jeongyeol Kwon, Ben Kretzu, Youliang Yu, Kaveh Hassani, Boris Vidolov, Yonathan Efroni

机构 * Meta AI ； Meta Superintelligence Labs（Meta 超智能实验室）； Columbia University（哥伦比亚大学）； Tel Aviv University（特拉维夫大学）

专题命中后训练与偏好优化：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2502.13177 2025-10-28 cs.LG cs.AI 86%

KL Penalty Control via Perturbation for Direct Preference Optimization

Sangkyu Lee, Janghoon Han, Hosung Song, Stanley Jungkyu Choi, Honglak Lee, Youngjae Yu

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published as a main conference track paper at NeurIPS 2025

2510.13512 2025-10-16 cs.LG cs.AI 86%

Offline and Online KL-Regularized RLHF under Differential Privacy

Yulian Wu, Rushil Thareja, Praneeth Vepakomma, Francesco Orabona

机构 * King Abdullah University of Science and Technology (KAUST)（卡布尔大学科学与技术学院）； Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）； Massachusetts Institute of Technology (MIT)（麻省理工学院）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2509.23564 2025-10-15 cs.AI cs.CL 86%

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment

Samuel Yeh, Sharon Li

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2025

2508.09138 2025-10-07 cs.CL cs.AI 86%

Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models

Wen Wang, Bozhen Fang, Chenchen Jing, Yongliang Shen, Yangyi Shen, Qiuyu Wang, Hao Ouyang, Hao Chen, Chunhua Shen

机构 * Zhejiang University（浙江大学）； Ant Group（蚂蚁集团）； Zhejiang University of Technology（浙江工业大学）； Stanford University（斯坦福大学）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI

Comments Project webpage: https://aim-uofa.github.io/dLLM-MidTruth

2509.25717 2025-10-01 cs.CV cs.CL cs.LG 86%

Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization

Xintong Li, Chuhan Wang, Junda Wu, Rohan Surana, Tong Yu, Julian McAuley, Jingbo Shang

机构 * University of California, San Diego（加州大学圣迭戈分校）； Adobe Research（Adobe研究）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2508.15845 2025-09-30 cs.CL cs.AI 86%

Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports

Chengbo Sun, Hui Yi Leong, Lei Li

机构 * University of Chicago（芝加哥大学）； University of Washington（华盛顿大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2509.21351 2025-09-29 cs.CV cs.AI cs.CL 86%

Random Direct Preference Optimization for Radiography Report Generation

Valentin Samokhin, Boris Shirokikh, Mikhail Goncharov, Dmitriy Umerenkov, Maksim Bobrin, Ivan Oseledets, Dmitry Dylov, Mikhail Belyaev

机构 * Artificial Intelligence Research Institute (AIRI)（人工智能研究院）； Skolkovo Institute of Science and Technology（斯克罗夫学院）； Institute for Information Transmission Problems（信息传输问题研究所）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2509.14172 2025-09-22 cs.LG cs.AI 86%

TGPO: Tree-Guided Preference Optimization for Robust Web Agent Reinforcement Learning

Ziyuan Chen, Zhenghui Zhao, Zhangye Han, Miancan Liu, Xianhang Ye, Yiqing Li, Hongbo Min, Jinkui Ren, Xiantao Zhang, Guitao Cao

机构 * East China Normal University（东华大学）； Alibaba Group（阿里巴巴集团）； University of Electronic Science and Technology of China（电子科技大学）； Wuhan University（武汉大学）； Sun Yat-sen University（中山大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2503.12301 2025-09-16 cs.LG cs.CL 86%

One Goal, Many Challenges: Robust Preference Optimization Amid Content-Aware and Multi-Source Noise

Amirabbas Afzali, Amirhossein Afsharrad, Seyed Shahabeddin Mousavi, Sanjay Lall

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2508.20181 2025-08-29 cs.CV cs.AI cs.CL cs.MM 86%

Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization

Alberto Compagnoni, Davide Caffagni, Nicholas Moratelli, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

机构 * University of Modena and Reggio Emilia（摩德纳和雷吉奥艾米利亚大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments BMVC 2025

2508.17000 2025-08-26 cs.CL cs.LG 86%

KL-Regularised Q-Learning: A Token-level Action-Value perspective on Online RLHF

Jason R Brown, Lennie Wells, Edward James Young, Sergio Bacallado

机构 * Computational and Biological Learning Group, Department of Engineering, University of Cambridge, Cambridge, UK（计算生物学学习组，工程系，剑桥大学，剑桥，英国）； Department of Computer Science and Technology, University of Cambridge, Cambridge, UK（计算机科学与技术系，剑桥大学，剑桥，英国）； Statistics Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, UK（统计实验室，纯粹数学与数学统计系，剑桥大学，英国）

专题命中后训练与偏好优化：RLHF(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2507.22789 2025-08-01 cs.LG cs.AI 86%

G-Core: A Simple, Scalable and Balanced RLHF Trainer

Junyu Wu, Weiming Chang, Xiaotao Liu, Guanyou He, Haoqiang Hong, Boqi Liu, Hongtao Tian, Tao Yang, Yunsheng Shi, Feng Lin, Ting Yao

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments I haven't received company approval yet, and I uploaded it by mistake

2506.15706 2025-06-23 cs.LG cs.AI 86%

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning

Yunze Lin

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2504.04524 2025-06-19 cs.LG cs.AI 86%

Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning

Xuerui Su, Shufang Xie, Guoqing Liu, Yingce Xia, Renqian Luo, Peiran Jin, Zhiming Ma, Yue Wang, Zun Wang, Yuting Liu

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 10pages

1. 后训练与偏好优化 4556 篇

Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLM

IB-GRPO: Aligning LLM-based Learning Path Recommendation with Educational Objectives via Indicator-Based Group Relative Policy Optimization

Evaluating LLM Behavior in Hiring: Implicit Weights, Fairness Across Groups, and Alignment with Human Preferences

GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation

Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback

Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection

Intelligently Weighting Multiple Reference Models for Direct Preference Optimization of LLMs

Comparative Analysis and Parametric Tuning of PPO, GRPO, and DAPO for LLM Reasoning Enhancement

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

Mitigating Length Bias in RLHF through a Causal Lens

AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress

Towards Reward Fairness in RLHF: From a Resource Allocation Perspective

The Polite Liar: Epistemic Pathology in Language Models

Detecting Prefix Bias in LLM-based Reward Models

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs

KL Penalty Control via Perturbation for Direct Preference Optimization

Offline and Online KL-Regularized RLHF under Differential Privacy

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment

Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models

Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization

Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports

Random Direct Preference Optimization for Radiography Report Generation

TGPO: Tree-Guided Preference Optimization for Robust Web Agent Reinforcement Learning

One Goal, Many Challenges: Robust Preference Optimization Amid Content-Aware and Multi-Source Noise

Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization

KL-Regularised Q-Learning: A Token-level Action-Value perspective on Online RLHF

G-Core: A Simple, Scalable and Balanced RLHF Trainer

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning

Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning