语言大模型 / LLM - arXivDaily 专题

2405.17382 2024-11-08 cs.LG cs.CL 86%

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Hyunseok Lee, Jihoon Tack, Jinwoo Shin

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference proceeding for NeurIPS 2024

2410.23726 2024-11-01 cs.AI cs.LG 86%

Towards Reliable Alignment: Uncertainty-aware RLHF

Debangshu Banerjee, Aditya Gopalan

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2408.09385 2024-10-31 cs.CL cs.AI 86%

Reward Difference Optimization For Sample Reweighting In Offline RLHF

Shiqi Wang, Zhengze Zhang, Rui Zhao, Fei Tan, Cam Tu Nguyen

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 findings

2410.19720 2024-10-28 cs.CL cs.AI 86%

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision

Shilong Li, Yancheng He, Hui Huang, Xingyuan Bu, Jiaheng Liu, Hangyu Guo, Weixun Wang, Jihao Gu, Wenbo Su, Bo Zheng

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments The first four authors contributed equally, 25 pages

2410.18585 2024-10-25 cs.AI cs.LG 86%

Aligning CodeLLMs with Direct Preference Optimization

Yibo Miao, Bofei Gao, Shanghaoran Quan, Junyang Lin, Daoguang Zan, Jiaheng Liu, Jian Yang, Tianyu Liu, Zhijie Deng

专题命中后训练与偏好优化：preference optimization(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

2410.09528 2024-10-25 cs.LG cs.AI 86%

Boosting Deductive Reasoning with Step Signals In RLHF

Jialian Li, Yipin Zhang, Wei Shen, Yuzi Yan, Jian Xie, Dong Yan

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2407.08639 2024-10-15 cs.AI cs.LG 86%

$β$-DPO: Direct Preference Optimization with Dynamic $β$

Junkang Wu, Yuexiang Xie, Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, Xiangnan He

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024

2410.01789 2024-10-03 cs.LG cs.AI 86%

Investigating on RLHF methodology

Alexey Kutalev, Sergei Markoff

专题命中后训练与偏好优化：RLHF(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 23 pages, 6 figures, 6 tables

2409.18618 2024-10-03 cs.CL cs.AI 86%

Model-based Preference Optimization in Abstractive Summarization without Human Feedback

Jaepill Choi, Kyubyung Chae, Jiwoo Song, Yohan Jo, Taesup Kim

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024

2405.16587 2024-10-03 cs.LG cs.AI cs.HC 86%

Cost-Effective Online Multi-LLM Selection with Versatile Reward Models

Xiangxiang Dai, Jin Li, Xutong Liu, Anqi Yu, John C. S. Lui

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 32 pages, 14 figures, conference

2403.03419 2024-10-01 cs.CL cs.AI 86%

Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization

Shitong Duan, Xiaoyuan Yi, Peng Zhang, Yan Liu, Zheng Liu, Tun Lu, Xing Xie, Ning Gu

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024(Findings)

2408.16751 2024-08-30 cs.CL cs.LG stat.ML 86%

A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models

Yi-Lin Tuan, William Yang Wang

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

2408.16032 2024-08-30 cs.LG cs.AI cs.IR 86%

An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders

Shuang Feng, Grace Feng

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2407.13709 2024-08-23 cs.CL cs.LG 86%

Understanding Reference Policies in Direct Preference Optimization

Yixin Liu, Pengfei Liu, Arman Cohan

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments GitHub Repo: https://github.com/yale-nlp/refdpo

2310.03708 2024-08-20 cs.LG cs.AI 86%

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization

Zhanhui Zhou, Jie Liu, Jing Shao, Xiangyu Yue, Chao Yang, Wanli Ouyang, Yu Qiao

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Findings of ACL 2024

2310.03716 2024-07-12 cs.CL cs.LG 86%

A Long Way to Go: Investigating Length Correlations in RLHF

Prasann Singhal, Tanya Goyal, Jiacheng Xu, Greg Durrett

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 21 pages, 13 figures, Accepted to COLM 2024

2406.06606 2024-07-09 cs.CL cs.AI 86%

Prototypical Reward Network for Data-Efficient RLHF

Jinghan Zhang, Xiting Wang, Yiqiao Jin, Changyu Chen, Xinhao Zhang, Kunpeng Liu

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024

2406.10977 2024-06-18 cs.CL cs.AI 86%

Toward Optimal LLM Alignments Using Two-Player Games

Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Hang Li, Yang Liu

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Our code is released at https://github.com/ruizheng20/gpo

2406.05587 2024-06-11 cs.CL cs.AI 86%

Creativity Has Left the Chat: The Price of Debiasing Language Models

Behnam Mohammadi

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

2402.14979 2024-06-07 cs.LG cs.CL stat.ME 86%

Optimizing Language Models for Human Preferences is a Causal Inference Problem

Victoria Lin, Eli Ben-Michael, Louis-Philippe Morency

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

Comments UAI 2024

2312.16682 2024-04-24 cs.CL cs.AI 86%

Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss

Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, Jason Weston

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2404.12150 2024-04-19 cs.LG cs.CL 86%

Aligning language models with human preferences

Tomasz Korbak

专题命中后训练与偏好优化：language model(title,abstract)；pretraining(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments PhD thesis

2404.08570 2024-04-15 cs.RO cs.AI cs.LG 86%

Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation

Hanlin Tian, Kethan Reddy, Yuxiang Feng, Mohammed Quddus, Yiannis Demiris, Panagiotis Angeloudis

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments 7 pages, 5 figures

2403.00409 2024-04-15 cs.LG cs.CL 86%

Provably Robust DPO: Aligning Language Models with Noisy Feedback

Sayak Ray Chowdhury, Anush Kini, Nagarajan Natarajan

专题命中后训练与偏好优化：language model(title,abstract)；SFT(abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

2404.00495 2024-04-02 cs.CL cs.AI 86%

Configurable Safety Tuning of Language Models with Synthetic Preference Data

Victor Gallego

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2312.16430 2024-03-26 cs.LG cs.AI 86%

Preference as Reward, Maximum Preference Optimization with Importance Sampling

Zaifan Jiang, Xing Huang, Chao Wei

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2403.07691 2024-03-15 cs.CL cs.AI 86%

ORPO: Monolithic Preference Optimization without Reference Model

Jiwoo Hong, Noah Lee, James Thorne

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Comments Preprint

2402.04792 2024-03-04 cs.AI cs.CL cs.HC 86%

Direct Language Model Alignment from Online AI Feedback

Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, Mathieu Blondel

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 9 figures, 4 tables

2402.16030 2024-02-27 cs.CL cs.AI 86%

Don't Forget Your Reward Values: Language Model Alignment via Value-based Calibration

Xin Mao, Feng-Lin Li, Huimin Xu, Wei Zhang, Anh Tuan Luu

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, Under review

2402.12222 2024-02-20 cs.CR cs.CL cs.LG cs.SE 86%

CovRL: Fuzzing JavaScript Engines with Coverage-Guided Reinforcement Learning for LLM-based Mutation

Jueon Eom, Seyeon Jeong, Taekyoung Kwon

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 14 pages, 4 figures, 9 tables, 2 listings

1. 后训练与偏好优化 4556 篇

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Towards Reliable Alignment: Uncertainty-aware RLHF

Reward Difference Optimization For Sample Reweighting In Offline RLHF

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision

Aligning CodeLLMs with Direct Preference Optimization

Boosting Deductive Reasoning with Step Signals In RLHF

$β$-DPO: Direct Preference Optimization with Dynamic $β$

Investigating on RLHF methodology

Model-based Preference Optimization in Abstractive Summarization without Human Feedback

Cost-Effective Online Multi-LLM Selection with Versatile Reward Models

Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization

A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models

An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders

Understanding Reference Policies in Direct Preference Optimization

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization

A Long Way to Go: Investigating Length Correlations in RLHF

Prototypical Reward Network for Data-Efficient RLHF

Toward Optimal LLM Alignments Using Two-Player Games

Creativity Has Left the Chat: The Price of Debiasing Language Models

Optimizing Language Models for Human Preferences is a Causal Inference Problem

Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss

Aligning language models with human preferences

Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation

Provably Robust DPO: Aligning Language Models with Noisy Feedback

Configurable Safety Tuning of Language Models with Synthetic Preference Data

Preference as Reward, Maximum Preference Optimization with Importance Sampling

ORPO: Monolithic Preference Optimization without Reference Model

Direct Language Model Alignment from Online AI Feedback

Don't Forget Your Reward Values: Language Model Alignment via Value-based Calibration

CovRL: Fuzzing JavaScript Engines with Coverage-Guided Reinforcement Learning for LLM-based Mutation