语言大模型 / LLM - arXivDaily 专题

2502.01930 2026-01-16 cs.LG cs.AI 91%

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization

通过分布鲁棒直接偏好优化实现鲁棒的大语言模型对齐

Zaiyan Xu, Sushil Vemuri, Kishan Panaganti, Dileep Kalathil, Rahul Jain, Deepak Ramachandran

机构 * Texas A&M University（德克萨斯大学）； California Institute of Technology（加州理工学院）； Tencent AI Lab（腾讯AI实验室）； Google DeepMind（谷歌DeepMind）

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

AI总结本文提出Wasserstein DPO和KLDPO算法，通过分布鲁棒优化解决LLM与人类偏好间的分布偏移问题，提升对齐性能。

Comments Accepted to NeurIPS 2025

2406.04208 2025-12-30 cs.LG cs.AI 91%

Aligning Agents like Large Language Models

像大型语言模型一样对齐代理

Adam Jelley, Yuhan Cao, Dave Bignell, Amos Storkey, Sam Devlin, Tabish Rashid

机构 * School of Informatics, University of Edinburgh, Edinburgh, United Kingdom（爱丁堡大学信息学院）； Micosoft Research, Cambridge, United Kingdom（微软研究院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

AI总结本文提出通过像训练大型语言模型一样训练代理，以实现更通用、稳健和对齐的行为，并通过3D视频游戏环境中的概念验证展示方法。

2508.06783 2025-12-11 cs.LG cs.AI cs.CR cs.IT math.IT 91%

PROPS: Progressively Private Self-alignment of Large Language Models

PROPS: 大型语言模型的逐步隐私自对齐

Noel Teku, Fengwei Tian, Payel Bhattacharjee, Souradip Chakraborty, Amrit Singh Bedi, Ravi Tandon

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

AI总结 PROPS通过多阶段隐私保护对齐框架，在保护偏好标签隐私的同时提升LLM对齐效果，实现更高的胜率。

Comments Accepted in the Transactions on Machine Learning Research (TMLR), 2025

Journal ref Transactions on ML Research (TMLR) 2025

2502.00313 2025-11-25 cs.GT cs.AI cs.CL cs.MA 91%

Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values

大语言模型中的分配公平性：评估与人类价值观的对齐

Hadi Hosseini, Samarth Khanna

机构 * Penn State University（宾夕法尼亚州立大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结本文研究了大语言模型在分配公平性方面的表现，发现其与人类价值观存在偏差，并探讨了提升对齐性的策略。

Comments Accepted at NeurIPS 2025

2511.15392 2025-11-20 cs.CL cs.AI 91%

DEPO: Dual-Efficiency Preference Optimization for LLM Agents

DEPO：用于LLM代理的双效偏好优化

Sirui Chen, Mengshi Zhao, Lei Xu, Yuying Zhao, Beier Zhu, Hanwang Zhang, Shengjie Zhao, Chaochao Lu

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

AI总结 DEPO通过双效偏好优化方法提升LLM代理的效率和性能，减少token和步骤消耗，同时在多个基准测试中表现优异。

Comments Accepted to AAAI 2026

2511.07483 2025-11-12 cs.AI cs.LG 91%

Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning

Qianxi He, Qingyu Ren, Shanzhe Lei, Xuhong Wang, Yingchun Wang

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）； Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（上海数据科学关键实验室，计算机科学学院，复旦大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；post-training(abstract)

2506.01183 2025-10-30 cs.LG cs.AI stat.ML 91%

Doubly Robust Alignment for Large Language Models

Erhan Xu, Kai Ye, Hongyi Zhou, Luhan Zhu, Francesco Quinzan, Chengchun Shi

机构 * Department of Statistics（统计系）； LSE London, UK（伦敦大学学院）； Department of Mathematics（数学系）； Tsinghua University（清华大学）； School of Design LCC, UAL London, UK（伦敦艺术大学设计学院）； Department of Engineering Science（工程科学系）； University of Oxford（牛津大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)

Comments Accepted to NeurIPS 2025

2504.09710 2025-10-14 cs.LG cs.CL 91%

DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training

Zhenting Wang, Guofeng Cui, Yu-Jhe Li, Kun Wan, Wentian Zhao

机构 * Rutgers University（罗杰斯大学）； Adobe Inc.（Adobe公司）

专题命中后训练与偏好优化：LLM(title,abstract)；post-training(title,abstract)；large language model(abstract)；language model(abstract)

2412.07192 2025-10-06 cs.CR cs.CL cs.LG 91%

PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips

Zachary Coalson, Jeonghyun Woo, Chris S. Lin, Joyce Qu, Yu Sun, Shiyang Chen, Lishan Yang, Gururaj Saileshwar, Prashant Nair, Bo Fang, Sanghyun Hong

机构 * Oregon State University（俄勒冈州立大学）； University of British Columbia（不列颠哥伦比亚大学）； University of Toronto（多伦多大学）； George Mason University（乔治·梅森大学）； Rutgers University（罗格斯大学）； University of Texas at Arlington（德克萨斯大学阿灵顿分校）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments Pre-print

2510.00911 2025-10-02 cs.LG cs.AI 91%

RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training

Tao Ren, Jinyang Jiang, Hui Yang, Wan Tian, Minhao Zou, Guanghao Li, Zishi Zhang, Qinghao Wang, Shentao Qin, Yanjun Zhao, Rui Tao, Hui Shao, Yijie Peng

机构 * Peking University（北京大学）

专题命中后训练与偏好优化：LLM(title,abstract)；post-training(title,abstract)；large language model(abstract)；language model(abstract)

2509.06759 2025-09-09 cs.LG cs.AI 91%

Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization

Thanh Thi Nguyen, Campbell Wilson, Janis Dalins

机构 * AiLECS Lab, Monash University Melbourne, Australia（墨尔本大学AiLECS实验室，澳大利亚）； AiLECS Lab, Monash University, Australia ICMEC Australia, Sydney, Australia（墨尔本大学AiLECS实验室，澳大利亚 ICMEC澳大利亚，悉尼，澳大利亚）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；pretraining(abstract)

Comments Accepted for publication in the Proceedings of the 8th International Conference on Algorithms, Computing and Artificial Intelligence (ACAI 2025)

2508.01930 2025-08-05 cs.CL cs.AI 91%

Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback

Tom S. Juzek, Zina B. Ward

机构 * Florida State University（佛罗里达州立大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments Accepted for publication in the Proceedings of the 5th Workshop on Bias and Fairness in AI (BIAS 2025) at ECML PKDD

2501.09620 2025-05-30 cs.LG cs.AI 91%

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment

Chaoqi Wang, Zhuokai Zhao, Yibo Jiang, Zhaorun Chen, Chen Zhu, Yuxin Chen, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, Hao Ma, Sinong Wang

机构 * Meta ； University of Chicago（芝加哥大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2505.09820 2025-05-16 cs.LG cs.CL cs.CR 91%

Adversarial Attack on Large Language Models using Exponentiated Gradient Descent

Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu

机构 * Department of Computer Science, Florida State University（计算机科学系，佛罗里达州立大学）； Department of Mathematics, Florida State University（数学系，佛罗里达州立大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments Accepted to International Joint Conference on Neural Networks (IJCNN) 2025

2503.22137 2025-03-31 cs.AI cs.LG 91%

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF

Syrine Belakaria, Joshua Kazdan, Charles Marx, Chris Cundy, Willie Neiswanger, Sanmi Koyejo, Barbara E. Engelhardt, Stefano Ermon

专题命中后训练与偏好优化：RLHF(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

2503.17965 2025-03-25 cs.CL cs.AI 91%

Understanding the Effects of RLHF on the Quality and Detectability of LLM-Generated Texts

Beining Xu, Arkaitz Zubiaga

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(title,abstract)；large language model(abstract)；language model(abstract)

Comments 14 pages, 3 figures

2410.14675 2025-03-18 cs.CL cs.AI 91%

To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts

Yukun Huang, Sanxing Chen, Hongyi Cai, Bhuwan Dhingra

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

2502.07072 2025-03-12 cs.CL cs.AI cs.SE 91%

IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models

Sayem Mohammad Imtiaz, Astha Singh, Fraol Batole, Hridesh Rajan

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments Accepted as full research paper at FSE'2025

2412.02674 2025-02-26 cs.CL cs.LG 91%

Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models

Yuda Song, Hanlin Zhang, Carson Eisenach, Sham Kakade, Dean Foster, Udaya Ghai

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments ICLR 2025; 41 pages, 19 figures

2406.14868 2025-02-25 cs.CL cs.LG 91%

Direct Multi-Turn Preference Optimization for Language Agents

Wentao Shi, Mengqi Yuan, Junkang Wu, Qifan Wang, Fuli Feng

专题命中后训练与偏好优化：preference optimization(title,abstract)；language agent(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by EMNLP 2024 Main

2405.19320 2025-02-20 cs.LG cs.AI stat.ML 91%

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Shicong Cen, Jincheng Mei, Katayoon Goshvadi, Hanjun Dai, Tong Yang, Sherry Yang, Dale Schuurmans, Yuejie Chi, Bo Dai

专题命中后训练与偏好优化：RLHF(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments ICLR 2025

2502.03147 2025-02-06 cs.CL cs.AI 91%

Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Models

Xumeng Wen, Shun Zheng, Zhen Xu, Yiming Sun, Jiang Bian

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments Preprint

2501.02869 2025-01-07 cs.CL cs.AI 91%

IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment

Yiming Zhang, Zheng Chang, Wentao Cai, MengXing Ren, Kang Yuan, Yining Sun, Zenghui Ding

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

2406.00832 2024-11-05 cs.CL cs.LG 91%

BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling

Lin Gui, Cristina Gârbacea, Victor Veitch

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2405.07135 2024-10-17 cs.LG cs.AI 91%

Post Training Quantization of Large Language Models with Microscaling Formats

Sayeh Sharify, Utkarsh Saxena, Zifei Xu, Wanzin Yazar, Ilya Soloveychik, Xin Wang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

2408.02599 2024-10-10 cs.CL cs.AI 91%

Progressively Label Enhancement for Large Language Model Alignment

Biao Liu, Ning Xu, Xin Geng

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2409.17791 2024-09-27 cs.CL cs.AI 91%

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness

Jian Li, Haojing Huang, Yujia Zhang, Pengfei Xu, Xi Chen, Rui Song, Lida Shi, Jingwen Wang, Hao Xu

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；RLHF(abstract)

Comments Accepted at EMNLP 2024 Findings

2406.09920 2024-09-25 cs.CL cs.AI 91%

Knowledge Editing in Language Models via Adapted Direct Preference Optimization

Amit Rozner, Barak Battash, Lior Wolf, Ofir Lindenbaum

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

Comments 9 pages, 4 figures

2311.09641 2024-06-21 cs.AI cs.CL cs.CR cs.HC 91%

RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models

Jiongxiao Wang, Junlin Wu, Muhao Chen, Yevgeniy Vorobeychik, Chaowei Xiao

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2406.12182 2024-06-19 cs.CL cs.AI 91%

Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models

Lulu Zhao, Weihao Zeng, Xiaofeng Shi, Hua Zhou, Donglin Hao, Yonghua Lin

专题命中后训练与偏好优化：LLM(title,abstract)；language model(title)；SFT(abstract)；RLHF(abstract)

1. 后训练与偏好优化 4535 篇

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization

Aligning Agents like Large Language Models

PROPS: Progressively Private Self-alignment of Large Language Models

Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values

DEPO: Dual-Efficiency Preference Optimization for LLM Agents

Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning

Doubly Robust Alignment for Large Language Models

DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training

PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips

RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training

Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization

Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment

Adversarial Attack on Large Language Models using Exponentiated Gradient Descent

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF

Understanding the Effects of RLHF on the Quality and Detectability of LLM-Generated Texts

To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts

IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models

Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models

Direct Multi-Turn Preference Optimization for Language Agents

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Models

IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment

BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling

Post Training Quantization of Large Language Models with Microscaling Formats

Progressively Label Enhancement for Large Language Model Alignment

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness

Knowledge Editing in Language Models via Adapted Direct Preference Optimization

RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models

Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models