语言大模型 / LLM - arXivDaily 专题

2505.16282 2025-05-23 cs.CV 80%

ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay

Fanbin Lu, Zhisheng Zhong, Shu Liu, Chi-Wing Fu, Jiaya Jia

机构 * The Chinese University of Hong Kong（香港中文大学）； SmartMore ； Hong Kong University of Science and Technology（香港理工大学）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2408.03618 2025-05-06 cs.CL cs.AI cs.LG 80%

A Logical Fallacy-Informed Framework for Argument Generation

Luca Mouchel, Debjit Paul, Shaobo Cui, Robert West, Antoine Bosselut, Boi Faltings

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

2504.14439 2025-04-22 cs.LG cs.AI cs.CL 80%

LoRe: Personalizing LLMs via Low-Rank Reward Modeling

Avinandan Bose, Zhihan Xiong, Yuejie Chi, Simon Shaolei Du, Lin Xiao, Maryam Fazel

机构 * FAIR at Meta（Meta 的 FAIR 部门）； University of Washington（华盛顿大学）； Carnegie Mellon University（卡内基梅隆大学）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2504.05810 2025-04-16 cs.CV 80%

PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning

Xinpeng Ding, Kui Zhang, Jianhua Han, Lanqing Hong, Hang Xu, Xiaomeng Li

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；SFT(abstract)；preference optimization(abstract)

2406.03199 2025-03-13 cs.CL cs.AI cs.LG 80%

Bayesian WeakS-to-Strong from Text Classification to Generation

Ziyun Cui, Ziyang Zhang, Guangzhi Sun, Wen Wu, Chao Zhang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ICLR2025

2502.18411 2025-03-04 cs.CV 80%

OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference

Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosong Cao, Weiyun Wang, Jiaqi Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, Haodong Duan, Hua Yang, Kai Chen

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；SFT(abstract)；preference optimization(abstract)

2502.20356 2025-02-28 cs.CL cs.AI cs.LG 80%

Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs

Kuan Lok Zhou, Jiayi Chen, Siddharth Suresh, Reuben Narad, Timothy T. Rogers, Lalit K Jain, Robert D Nowak, Bob Mankoff, Jifan Zhang

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2409.04421 2025-01-20 cs.CL cs.AI cs.LG 80%

RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs

Jiaxing Wu, Lin Ning, Luyang Liu, Harrison Lee, Neo Wu, Chao Wang, Sushant Prakash, Shawn O'Banion, Bradley Green, Jun Xie

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments AAAI 2025

2310.05857 2025-01-07 cs.CL cs.AI cs.LG 80%

Improving Summarization with Human Edits

Zonghai Yao, Benjamin J Schloss, Sai P. Selvaraj

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Proceedings of the Main Conference on Empirical Methods in Natural Language Processing (EMNLP) 2023

2406.10522 2024-12-19 cs.LG cs.AI cs.CL 80%

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Lok Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy Rogers, Kevin Jamieson, Robert Mankoff, Robert Nowak

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.12535 2024-12-17 cs.LG cs.AI cs.CL 80%

Is There No Such Thing as a Bad Question? H4R: HalluciBot For Ratiocination, Rewriting, Ranking, and Routing

William Watson, Nicole Cho, Nishan Srishankar

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at The 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025)

2406.16235 2024-11-11 cs.CL cs.AI cs.CR cs.LG 80%

Preference Tuning For Toxicity Mitigation Generalizes Across Languages

Xiaochen Li, Zheng-Xin Yong, Stephen H. Bach

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Findings of EMNLP 2024

2401.16635 2024-10-23 cs.LG cs.AI cs.CL 80%

Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble

Shun Zhang, Zhenfang Chen, Sunli Chen, Yikang Shen, Zhiqing Sun, Chuang Gan

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2407.17546 2024-07-26 cs.LG cs.AI cs.CL 80%

Exploring Domain Robust Lightweight Reward Models based on Router Mechanism

Hyuk Namgoong, Jeesu Jung, Sangkeun Jung, Yoonhyung Roh

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This paper is accepted for ACL 2024

2406.01462 2024-07-17 cs.LG cs.AI cs.CL 80%

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage

Yuda Song, Gokul Swamy, Aarti Singh, J. Andrew Bagnell, Wen Sun

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.13228 2024-07-04 cs.CL cs.AI cs.LG 80%

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive

Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, Colin White

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.03746 2024-06-18 cs.CV 80%

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang, Jonghyun Choi

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Comments ACL 2024

2406.08673 2024-06-14 cs.CL cs.AI cs.LG 80%

HelpSteer2: Open-source dataset for training top-performing reward models

Zhilin Wang, Yi Dong, Olivier Delalleau, Jiaqi Zeng, Gerald Shen, Daniel Egert, Jimmy J. Zhang, Makesh Narsimhan Sreedhar, Oleksii Kuchaiev

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.02479 2024-06-11 cs.LG cs.AI cs.CL cs.HC 80%

BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback

Gaurav Pandey, Yatin Nandwani, Tahira Naseem, Mayank Mishra, Guangxuan Xu, Dinesh Raghu, Sachindra Joshi, Asim Munawar, Ramón Fernandez Astudillo

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICML 2024 (main conference)

2405.11165 2024-05-30 cs.CV 80%

Automated Multi-level Preference for MLLMs

Mengxi Zhang, Wenhao Wu, Yu Lu, Yuxin Song, Kang Rong, Huanjin Yao, Jianbo Zhao, Fanglong Liu, Yifan Sun, Haocheng Feng, Jingdong Wang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments Preprint

2311.14455 2024-04-30 cs.AI cs.CL cs.CR cs.LG 80%

Universal Jailbreak Backdoors from Poisoned Human Feedback

Javier Rando, Florian Tramèr

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted as conference paper in ICLR 2024

2404.07491 2024-04-12 cs.SE 80%

Neural Fault Injection: Generating Software Faults from Natural Language

Domenico Cotroneo, Pietro Liguori

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments 2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Disrupt

2309.10691 2024-03-13 cs.CL cs.AI cs.LG 80%

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024. Code is available on our project website: https://xingyaoww.github.io/mint-bench

2402.01694 2024-02-06 cs.CL cs.AI cs.LG 80%

ARGS: Alignment as Reward-Guided Search

Maxim Khanov, Jirayu Burapacheep, Yixuan Li

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024

2311.14115 2024-01-11 cs.LG cs.AI cs.CL 80%

A density estimation perspective on learning from pairwise human preferences

Vincent Dumoulin, Daniel D. Johnson, Pablo Samuel Castro, Hugo Larochelle, Yann Dauphin

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2305.14387 2024-01-09 cs.LG cs.AI cs.CL 80%

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, Tatsunori B. Hashimoto

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Spotlight at NeurIPS 2023

2310.16048 2023-10-25 cs.AI cs.CL cs.CY cs.HC cs.LG 80%

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications

Abhilash Mishra

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, no figures

2307.15217 2023-09-12 cs.AI cs.CL cs.LG 80%

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, Tony Wang, Samuel Marks, Charbel-Raphaël Segerie, Micah Carroll, Andi Peng, Phillip Christoffersen, Mehul Damani, Stewart Slocum, Usman Anwar, Anand Siththaranjan, Max Nadeau, Eric J. Michaud, Jacob Pfau, Dmitrii Krasheninnikov, Xin Chen, Lauro Langosco, Peter Hase, Erdem Bıyık, Anca Dragan, David Krueger, Dorsa Sadigh, Dylan Hadfield-Menell

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

2305.12219 2023-05-26 cs.LG cs.AI cs.CL 80%

Collaborative Development of NLP models

Fereshte Khani, Marco Tulio Ribeiro

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

2507.09973 2025-07-15 cs.CL cs.AI 80%

Tiny Reward Models

Sarah Pan

机构 * Massachusetts Institute of Technology（麻省理工学院）

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 2025 ICML Efficient Systems for Foundation Models Workshop

1. 后训练与偏好优化 4541 篇

ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay

A Logical Fallacy-Informed Framework for Argument Generation

LoRe: Personalizing LLMs via Low-Rank Reward Modeling

PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning

Bayesian WeakS-to-Strong from Text Classification to Generation

OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference

Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs

RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs

Improving Summarization with Human Edits

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Is There No Such Thing as a Bad Question? H4R: HalluciBot For Ratiocination, Rewriting, Ranking, and Routing

Preference Tuning For Toxicity Mitigation Generalizes Across Languages

Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble

Exploring Domain Robust Lightweight Reward Models based on Router Mechanism

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

HelpSteer2: Open-source dataset for training top-performing reward models

BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback

Automated Multi-level Preference for MLLMs

Universal Jailbreak Backdoors from Poisoned Human Feedback

Neural Fault Injection: Generating Software Faults from Natural Language

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

ARGS: Alignment as Reward-Guided Search

A density estimation perspective on learning from pairwise human preferences

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

AI Alignment and Social Choice: Fundamental Limitations and Policy Implications

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

Collaborative Development of NLP models

Tiny Reward Models