语言大模型 / LLM - arXivDaily 专题

2503.20105 2025-03-27 cs.AI cs.RO 83%

Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations

Ran Tian, Kratarth Goel

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；分类 cs.AI

Comments ICLR 2025 Spotlight

2503.10480 2025-03-14 cs.CL cs.CV cs.RO 83%

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, Xipeng Qiu

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2410.18640 2025-03-07 cs.CL 83%

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Wenhong Zhu, Zhiwei He, Xiaofeng Wang, Pengfei Liu, Rui Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

Comments ICLR 2025(Spotlight)

2502.20847 2025-03-03 cs.LG 83%

Gradient Imbalance in Direct Preference Optimization

Qinwei Ma, Jingzhe Shi, Can Jin, Jenq-Neng Hwang, Serge Belongie, Lei Li

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.LG

Comments 15 pages, 2 figures

2412.03187 2025-02-27 cs.CL 83%

Weighted-Reward Preference Optimization for Implicit Model Fusion

Ziyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi, Xiaojun Quan

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to ICLR 2025

2502.14356 2025-02-21 cs.CL 83%

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

Huimin Xu, Xin Mao, Feng-Lin Li, Xiaobao Wu, Wang Chen, Wei Zhang, Anh Tuan Luu

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2411.14421 2024-11-27 cs.LG 83%

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Shourya Bose, Yijiang Li, Amy Van Sant, Yu Zhang, Kibaek Kim

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(abstract)；分类 cs.LG

Comments NeurIPS 2024 Workshop on Time Series in the Age of Large Models

2411.09661 2024-11-15 cs.CL 83%

Adaptive Decoding via Latent Preference Optimization

Shehzaad Dhuliawala, Ilia Kulikov, Ping Yu, Asli Celikyilmaz, Jason Weston, Sainbayar Sukhbaatar, Jack Lanchantin

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2406.09215 2024-11-08 cs.IR cs.AI 83%

On Softmax Direct Preference Optimization for Recommendation

Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, Tat-Seng Chua

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.AI

Comments NeurIPS 2024

2410.17055 2024-10-24 cs.LG stat.ML 83%

Optimal Design for Reward Modeling in RLHF

Antoine Scheid, Etienne Boursier, Alain Durmus, Michael I. Jordan, Pierre Ménard, Eric Moulines, Michal Valko

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.LG

2307.00279 2024-10-24 cs.CL 83%

Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

Beatriz Borges, Niket Tandon, Tanja Käser, Antoine Bosselut

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments EMNLP 2024; 9 pages, 3 figures

2410.17389 2024-10-24 cs.AI 83%

Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models

Muhan Lin, Shuyang Shi, Yue Guo, Behdad Chalaki, Vaishnav Tadiparthi, Ehsan Moradi Pari, Simon Stepputtis, Joseph Campbell, Katia Sycara

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments 13 pages, 8 figures, The 2024 Conference on Empirical Methods in Natural Language Processing

2410.16184 2024-10-22 cs.CL 83%

RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style

Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, Juanzi Li

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

2410.16027 2024-10-22 cs.CL 83%

ComPO: Community Preferences for Language Model Personalization

Sachin Kumar, Chan Young Park, Yulia Tsvetkov, Noah A. Smith, Hannaneh Hajishirzi

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

2410.12194 2024-10-17 cs.CL 83%

Negative-Prompt-driven Alignment for Generative Language Model

Shiqi Qiao, Ning Xv, Biao Liu, Xin Geng

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2408.12163 2024-10-07 cs.CL 83%

Preference-Guided Reflective Sampling for Aligning Language Models

Hai Ye, Hwee Tou Ng

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments EMNLP2024, main

2408.15630 2024-08-29 cs.SE cs.AI 83%

CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation

Pooja Aggarwal, Oishik Chatterjee, Ting Dai, Prateeti Mohapatra, Brent Paulovicks, Brad Blancett, Arthur De Magalhaes

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2404.12358 2024-08-14 cs.LG 83%

From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function

Rafael Rafailov, Joey Hejna, Ryan Park, Chelsea Finn

专题命中后训练与偏好优化：language model(title)；RLHF(abstract)；preference optimization(abstract)；分类 cs.LG

Comments COLM 2024

2407.00215 2024-07-02 cs.SE cs.LG 83%

LLM Critics Help Catch LLM Bugs

Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, Jan Leike

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(abstract)；分类 cs.LG

2406.07168 2024-06-12 cs.CL 83%

Teaching Language Models to Self-Improve by Learning from Language Feedback

Chi Hu, Yimin Hu, Hang Cao, Tong Xiao, Jingbo Zhu

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Findings of ACL 2024

2402.14688 2024-06-04 cs.LG 83%

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models

Kenneth Li, Samy Jelassi, Hugh Zhang, Sham Kakade, Martin Wattenberg, David Brandfonbrener

专题命中后训练与偏好优化：language model(title,abstract)；prompting(abstract)；分类 cs.LG

2305.14718 2024-04-23 cs.CL 83%

Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models

Ashutosh Baheti, Ximing Lu, Faeze Brahman, Ronan Le Bras, Maarten Sap, Mark Riedl

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments published at ICLR 2024

2404.09824 2024-04-16 cs.CL 83%

Impact of Preference Noise on the Alignment Performance of Generative Language Models

Yang Gao, Dana Alon, Donald Metzler

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

2312.07551 2023-12-14 cs.CL 83%

Language Model Alignment with Elastic Reset

Michael Noukhovitch, Samuel Lavoie, Florian Strub, Aaron Courville

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments Published at NeurIPS 2023

2306.01693 2023-10-31 cs.CL 83%

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, Hannaneh Hajishirzi

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments NeurIPS 2023 camera-ready

2310.16681 2023-10-26 cs.CL 83%

BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?

Xingmeng Zhao, Tongnian Wang, Sheri Osborn, Anthony Rios

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments Accepted to BabyLM workshop at CoNLL

2204.13498 2022-04-29 cs.CL 83%

Post-Training Dialogue Summarization using Pseudo-Paraphrasing

Qi Jia, Yizhu Liu, Haifeng Tang, Kenny Q. Zhu

专题命中后训练与偏好优化：post-training(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Findings of NAACL 2022

2106.02327 2021-06-07 cs.CL 83%

Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene

Ruikun Luo, Guanhuan Huang, Xiaojun Quan

专题命中后训练与偏好优化：post-training(title,abstract)；language model(abstract)；分类 cs.CL

1904.02232 2019-05-07 cs.CL 83%

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

Hu Xu, Bing Liu, Lei Shu, Philip S. Yu

专题命中后训练与偏好优化：post-training(title,abstract)；language model(abstract)；分类 cs.CL

Comments accepted by NAACL 2019

2603.17378 2026-08-24 cs.LG cs.AI 版本更新 82%

Efficient Exploration at Scale

大规模高效探索

Seyed Mohammad Asghari, Chris Chute, Vikranth Dwaracherla, Xiuyuan Lu, Mehdi Jafarnia, Victor Minden, Zheng Wen, Benjamin Van Roy

机构 * Google（谷歌）； DeepMind（深度思维）

专题命中后训练与偏好优化：RLHF(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结本文提出一种在线学习算法，通过增量更新奖励和语言模型提升强化学习从人类反馈中的数据效率，采用小幅度正向激励、信念神经网络和信息导向探索等机制，实验表明在少于20k标签下达到200k标签的性能，预计1M标签可匹配1B标签的性能。

1. 后训练与偏好优化 4556 篇

Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Gradient Imbalance in Direct Preference Optimization

Weighted-Reward Preference Optimization for Implicit Model Fusion

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Adaptive Decoding via Latent Preference Optimization

On Softmax Direct Preference Optimization for Recommendation

Optimal Design for Reward Modeling in RLHF

Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models

RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style

ComPO: Community Preferences for Language Model Personalization

Negative-Prompt-driven Alignment for Generative Language Model

Preference-Guided Reflective Sampling for Aligning Language Models

CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation

From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function

LLM Critics Help Catch LLM Bugs

Teaching Language Models to Self-Improve by Learning from Language Feedback

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models

Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models

Impact of Preference Noise on the Alignment Performance of Generative Language Models

Language Model Alignment with Elastic Reset

Fine-Grained Human Feedback Gives Better Rewards for Language Model Training

BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?

Post-Training Dialogue Summarization using Pseudo-Paraphrasing

Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

Efficient Exploration at Scale