语言大模型 / LLM - arXivDaily 专题

2503.03122 2025-05-22 cs.CL cs.AI 73%

The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models

Zichao Li, Xueru Wen, Jie Lou, Yuqiu Ji, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments ICML 2025

2505.08295 2025-05-14 cs.LG cs.AI 73%

A Practical Introduction to Deep Reinforcement Learning

Yinghan Sun, Hongxi Wang, Hua Chen, Wei Zhang

机构 * Southern University of Science and Technology（南方科技大学）； Zhejiang University-University of Illinois Urbana-Champaign Institute（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2312.08365 2025-05-12 cs.LG cs.AI 73%

An Invitation to Deep Reinforcement Learning

Bernhard Jaeger, Andreas Geiger

机构 * University of Tübingen, Tübingen AI Center（图宾根大学，图宾根人工智能中心）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published at Foundations and Trends in Optimization

Journal ref Foundations and Trends in Optimization, 2024, Vol. 7: No. 1, pp 1-80

2505.01435 2025-05-06 cs.IR cs.CL cs.DC cs.LG 73%

AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine

Carlo Siebenschuh, Kyle Hippe, Ozan Gokdemir, Alexander Brace, Arham Khan, Khalid Hossain, Yadu Babuji, Nicholas Chia, Venkatram Vishwanath, Rick Stevens, Arvind Ramanathan, Ian Foster, Robert Underwood

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

Comments This paper has been accepted at the The Eighth Annual Conference on Machine Learning and Systems (MLSys 2025)

2504.10559 2025-04-16 cs.LG cs.AI 73%

Efficient Process Reward Model Training via Active Learning

Keyu Duan, Zichen Liu, Xin Mao, Tianyu Pang, Changyu Chen, Qiguang Chen, Michael Qizhe Shieh, Longxu Dou

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 15 pages, 4 figures

2503.08295 2025-04-10 cs.LG cs.AI 73%

Preference-Based Alignment of Discrete Diffusion Models

Umberto Borso, Davide Paglieri, Jude Wells, Tim Rocktäschel

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

2503.13538 2025-03-19 cs.LG cs.AI 73%

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences

Siliang Zeng, Yao Liu, Huzefa Rangwala, George Karypis, Mingyi Hong, Rasool Fakoor

专题命中后训练与偏好优化：LLM(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2503.05453 2025-03-10 cs.LG cs.AI 73%

Soft Policy Optimization: Online Off-Policy RL for Sequence Models

Taco Cohen, David W. Zhang, Kunhao Zheng, Yunhao Tang, Remi Munos, Gabriel Synnaeve

专题命中后训练与偏好优化：language model(abstract)；post-training(abstract)；分类 cs.AI、cs.LG

2503.00295 2025-03-04 cs.CL cs.LG 73%

Robust Multi-Objective Preference Alignment with Online DPO

Raghav Gupta, Ryan Sullivan, Yunxuan Li, Samrat Phatale, Abhinav Rastogi

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments AAAI 2025 - AI Alignment Track

2404.09656 2025-02-26 cs.LG cs.CL 73%

Learn Your Reference Model for Real Good Alignment

Alexey Gorbatovski, Boris Shaposhnikov, Alexey Malakhov, Nikita Surnachev, Yaroslav Aksenov, Ian Maksimov, Nikita Balagansky, Daniil Gavrilov

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2502.14204 2025-02-21 cs.CL cs.AI 73%

On-the-fly Preference Alignment via Principle-Guided Decoding

Mingye Zhu, Yi Liu, Lei Zhang, Junbo Guo, Zhendong Mao

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025

2502.06861 2025-02-12 cs.LG cs.AI 73%

Design Considerations in Offline Preference-based RL

Alekh Agarwal, Christoph Dann, Teodor V. Marinov

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2409.00352 2025-02-11 cs.CL cs.LG 73%

Does Alignment Tuning Really Break LLMs' Internal Confidence?

Hongseok Oh, Wonseok Hwang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Presented at the BlackboxNLP Workshop at EMNLP 2024 (Poster)

2502.01652 2025-02-05 cs.LG cs.AI 73%

Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimization

Soham Sane

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 11 Pages, 18 Equations, 1 Table

2501.15109 2025-01-28 cs.LG cs.AI 73%

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning

Nirav Diwan, Tolga Ergen, Dongsub Shim, Honglak Lee

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

2412.16475 2024-12-24 cs.LG cs.AI stat.ML 73%

When Can Proxies Improve the Sample Complexity of Preference Learning?

Yuchen Zhu, Daniel Augusto de Souza, Zhengyan Shi, Mengyue Yang, Pasquale Minervini, Alexander D'Amour, Matt J. Kusner

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2412.13862 2024-12-19 cs.LG cs.CL 73%

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model

Yuzhong Hong, Hanshan Zhang, Junwei Bao, Hongfei Jiang, Yang Song

专题命中后训练与偏好优化：LLM(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

2412.13746 2024-12-19 cs.CL cs.AI cs.IR 73%

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment

Zhuoran Jin, Hongbang Yuan, Tianyi Men, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

专题命中后训练与偏好优化：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 12 figures, 6 tables

2412.03822 2024-12-06 cs.CL cs.AI 73%

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization

Vishakh Padmakumar, Chuanyang Jin, Hannah Rose Kirk, He He

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2410.16509 2024-10-23 cs.CL cs.LG 73%

Learning from others' mistakes: Finetuning machine translation models with span-level error annotations

Lily H. Zhang, Hamid Dadkhahi, Mara Finkelstein, Firas Trabelsi, Jiaming Luo, Markus Freitag

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

2410.08134 2024-10-11 cs.LG cs.AI 73%

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

Jarrid Rector-Brooks, Mohsin Hasan, Zhangzhi Peng, Zachary Quinn, Chenghao Liu, Sarthak Mittal, Nouha Dziri, Michael Bronstein, Yoshua Bengio, Pranam Chatterjee, Alexander Tong, Avishek Joey Bose

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2410.05328 2024-10-10 cs.LG cs.AI 73%

Reward Learning From Preference With Ties

Jinsong Liu, Dongdong Ge, Ruihao Zhu

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2409.05258 2024-09-10 cs.LG cs.AI cs.CV 73%

Towards Automated Machine Learning Research

Shervin Ardeshir

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2409.01552 2024-09-04 cs.CL cs.AI 73%

Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Zhuo Li, Yuhao Du, Jinpeng Hu, Xiang Wan, Anningzhe Gao

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2403.18742 2024-08-08 cs.LG cs.AI 73%

Understanding the Learning Dynamics of Alignment with Human Feedback

Shawn Im, Yixuan Li

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2406.01013 2024-06-21 cs.LG cs.CL 73%

Scalable Ensembling For Mitigating Reward Overoptimisation

Ahmed M. Ahmed, Rafael Rafailov, Stepan Sharkov, Xuechen Li, Sanmi Koyejo

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

2406.10278 2024-06-18 cs.CL cs.AI 73%

Prompt-Based Length Controlled Generation with Multiple Control Types

Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 findings. arXiv admin note: text overlap with arXiv:2308.12030

2401.06688 2024-06-07 cs.CL cs.LG 73%

Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation

Giorgos Vernikos, Andrei Popescu-Belis

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ACL 2024

2405.00438 2024-05-02 cs.LG cs.CL 73%

MetaRM: Shifted Distributions Alignment via Meta-Learning

Shihan Dou, Yan Liu, Enyu Zhou, Tianlong Li, Haoxiang Jia, Limao Xiong, Xin Zhao, Junjie Ye, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments 11 pages, 6 figures. arXiv admin note: text overlap with arXiv:2401.06080

2404.04932 2024-04-09 cs.CL cs.AI 73%

Towards Understanding the Influence of Reward Margin on Preference Model Performance

Bowen Qin, Duanyu Feng, Xi Yang

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

1. 后训练与偏好优化 4535 篇

The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models

A Practical Introduction to Deep Reinforcement Learning

An Invitation to Deep Reinforcement Learning

AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine

Efficient Process Reward Model Training via Active Learning

Preference-Based Alignment of Discrete Diffusion Models

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences

Soft Policy Optimization: Online Off-Policy RL for Sequence Models

Robust Multi-Objective Preference Alignment with Online DPO

Learn Your Reference Model for Real Good Alignment

On-the-fly Preference Alignment via Principle-Guided Decoding

Design Considerations in Offline Preference-based RL

Does Alignment Tuning Really Break LLMs' Internal Confidence?

Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimization

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning

When Can Proxies Improve the Sample Complexity of Preference Learning?

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization

Learning from others' mistakes: Finetuning machine translation models with span-level error annotations

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

Reward Learning From Preference With Ties

Towards Automated Machine Learning Research

Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Understanding the Learning Dynamics of Alignment with Human Feedback

Scalable Ensembling For Mitigating Reward Overoptimisation

Prompt-Based Length Controlled Generation with Multiple Control Types

Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation

MetaRM: Shifted Distributions Alignment via Meta-Learning

Towards Understanding the Influence of Reward Margin on Preference Model Performance