语言大模型 / LLM - arXivDaily 专题

2412.17696 2025-03-28 cs.CL 70%

Understanding the Logic of Direct Preference Alignment through Logic

Kyle Richardson, Vivek Srikumar, Ashish Sabharwal

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2503.20271 2025-03-27 cs.CV cs.CL 70%

ViLBench: A Suite for Vision-Language Process Reward Modeling

Haoqin Tu, Weitao Feng, Hardy Chen, Hui Liu, Xianfeng Tang, Cihang Xie

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2407.16008 2025-03-18 cs.CL 70%

Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generation

Jiaming Shen, Ran Xu, Yennie Jun, Zhen Qin, Tianqi Liu, Carl Yang, Yi Liang, Simon Baumgartner, Michael Bendersky

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ICLR 2025 SSI-FM version

2503.10228 2025-03-14 cs.LG 70%

Policy Teaching via Data Poisoning in Learning from Human Preferences

Andi Nika, Jonathan Nöther, Debmalya Mandal, Parameswaran Kamalaruban, Adish Singla, Goran Radanović

专题命中后训练与偏好优化：RLHF(abstract)；preference optimization(abstract)；分类 cs.LG

Comments In AISTATS 2025

2503.10215 2025-03-14 cs.AI cs.GT 70%

Adaptive Preference Aggregation

Benjamin Heymann

专题命中后训练与偏好优化：foundation model(abstract)；RLHF(abstract)；分类 cs.AI

2503.08663 2025-03-12 cs.RO cs.AI cs.CV cs.CY cs.HC 70%

Generating Robot Constitutions & Benchmarks for Semantic Safety

Pierre Sermanet, Anirudha Majumdar, Alex Irpan, Dmitry Kalashnikov, Vikas Sindhwani

专题命中后训练与偏好优化：language model(abstract)；foundation model(abstract)；分类 cs.AI

2410.02451 2025-03-11 cs.AI 70%

Strong Preferences Affect the Robustness of Preference Models and Value Alignment

Ziwei Xu, Mohan Kankanhalli

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 21 Pages. Accepted by ICLR 2025

2410.04166 2025-03-10 cs.LG stat.ML 70%

Learning from negative feedback, or positive feedback or both

Abbas Abdolmaleki, Bilal Piot, Bobak Shahriari, Jost Tobias Springenberg, Tim Hertweck, Rishabh Joshi, Junhyuk Oh, Michael Bloesch, Thomas Lampe, Nicolas Heess, Jonas Buchli, Martin Riedmiller

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.LG

2502.02954 2025-03-07 cs.LG 70%

Direct Distributional Optimization for Provable Alignment of Diffusion Models

Ryotaro Kawata, Kazusato Oko, Atsushi Nitanda, Taiji Suzuki

专题命中后训练与偏好优化：RLHF(abstract)；preference optimization(abstract)；分类 cs.LG

2410.11817 2025-03-04 cs.CV cs.LG cs.MM 70%

Improving Long-Text Alignment for Text-to-Image Diffusion Models

Luping Liu, Chao Du, Tianyu Pang, Zehan Wang, Chongxuan Li, Dong Xu

专题命中后训练与偏好优化：foundation model(abstract)；preference optimization(abstract)；分类 cs.LG

Journal ref International Conference on Learning Representations (ICLR 2025)

2409.13156 2025-02-28 cs.CL 70%

RRM: Robust Reward Model Training Mitigates Reward Hacking

Tianqi Liu, Wei Xiong, Jie Ren, Lichang Chen, Junru Wu, Rishabh Joshi, Yang Gao, Jiaming Shen, Zhen Qin, Tianhe Yu, Daniel Sohn, Anastasiia Makarova, Jeremiah Liu, Yuan Liu, Bilal Piot, Abe Ittycheriah, Aviral Kumar, Mohammad Saleh

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted in ICLR 2025

2502.14361 2025-02-21 cs.AI cs.IR 70%

Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning

Jiachen Zhu, Congmin Zheng, Jianghao Lin, Kounianhua Du, Ying Wen, Yong Yu, Jun Wang, Weinan Zhang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

2502.11250 2025-02-18 cs.CL 70%

Uncertainty-Aware Step-wise Verification with Generative Reward Models

Zihuiwen Ye, Luckeciano Carvalho Melo, Younesse Kaddar, Phil Blunsom, Sam Staton, Yarin Gal

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2502.01715 2025-02-05 cs.SE cs.AI 70%

Process-Supervised Reinforcement Learning for Code Generation

Yufan Ye, Ting Zhang, Wenbin Jiang, Hua Huang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

2410.17621 2025-02-05 cs.AI 70%

Process Supervision-Guided Policy Optimization for Code Generation

Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang, Lin Yan

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 15 pages, 8 figures

2408.12109 2025-02-03 cs.CV cs.CL 70%

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data

Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Murun Yang, Qiaozhi He, Tong Xiao, Chunliang Zhang, Tongran Liu, Quan Du, Di Yang, Jingbo Zhu

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by AAAI 2025

2407.04185 2025-01-09 cs.CL 70%

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

Shujun Liu, Xiaoyu Shen, Yuhang Lai, Siyuan Wang, Shengbin Yue, Zengfeng Huang, Xuanjing Huang, Zhongyu Wei

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2501.03995 2025-01-08 cs.LG cs.CV cs.IR cs.IT math.IT 70%

RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance

Matin Mortaheb, Mohammad A. Amir Khojastepour, Srimat T. Chakradhar, Sennur Ulukus

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.LG

2501.00911 2025-01-03 cs.LG 70%

Aligning LLMs with Domain Invariant Reward Models

David Wu, Sanjiban Choudhury

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.LG

2412.12888 2024-12-19 cs.CV cs.AI 70%

ArtAug: Enhancing Text-to-Image Generation through Synthesis-Understanding Interaction

Zhongjie Duan, Qianyi Zhao, Cen Chen, Daoyuan Chen, Wenmeng Zhou, Yaliang Li, Yingda Chen

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 18 pages, 8 figures

2407.16574 2024-12-10 cs.CL 70%

TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback

Eunseop Yoon, Hee Suk Yoon, SooHwan Eom, Gunsoo Han, Daniel Wontae Nam, Daejin Jo, Kyoung-Woon On, Mark A. Hasegawa-Johnson, Sungwoong Kim, Chang D. Yoo

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.CL

Comments ACL2024 Findings

2412.05210 2024-12-09 cs.CL 70%

Evaluating and Aligning CodeLLMs on Human Preference

Jian Yang, Jiaxi Yang, Ke Jin, Yibo Miao, Lei Zhang, Liqun Yang, Zeyu Cui, Yichang Zhang, Binyuan Hui, Junyang Lin

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2411.00722 2024-11-04 cs.LG 70%

Token-level Proximal Policy Optimization for Query Generation

Yichen Ouyang, Lu Wang, Fangkai Yang, Pu Zhao, Chenghua Huang, Jianfeng Liu, Bochen Pang, Yaming Yang, Yuefeng Zhan, Hao Sun, Qingwei Lin, Saravan Rajmohan, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 10 pages

2407.13887 2024-10-25 cs.CL 70%

Learning Goal-Conditioned Representations for Language Reward Models

Vaskar Nath, Dylan Slack, Jeff Da, Yuntao Ma, Hugh Zhang, Spencer Whitehead, Sean Hendryx

专题命中后训练与偏好优化：language model(abstract)；RLHF(abstract)；分类 cs.CL

2405.20850 2024-10-21 cs.CL 70%

Improving Reward Models with Synthetic Critiques

Zihuiwen Ye, Fraser Greenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, Matthias Gallé

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

2410.08710 2024-10-17 cs.LG stat.ML 70%

Preferential Normalizing Flows

Petrus Mikkola, Luigi Acerbi, Arto Klami

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 29 pages, 18 figures, Accepted at NeurIPS2024

2405.07726 2024-10-17 cs.CL 70%

Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing

Letian Peng, Jingbo Shang

专题命中后训练与偏好优化：LLM(abstract)；preference optimization(abstract)；分类 cs.CL

Comments NeurIPS2024

2410.05496 2024-10-10 cs.AI cs.GT 70%

Intuitions of Compromise: Utilitarianism vs. Contractualism

Jared Moore, Yejin Choi, Sydney Levine

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.AI

2406.15352 2024-10-07 cs.CL 70%

A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning Stick

Nishant Balepur, Matthew Shu, Alexander Hoyle, Alison Robey, Shi Feng, Seraphina Goldfarb-Tarrant, Jordan Boyd-Graber

专题命中后训练与偏好优化：LLM(abstract)；preference optimization(abstract)；分类 cs.CL

Comments EMNLP 2024

2310.03304 2024-10-07 cs.CL 70%

Learning Personalized Alignment for Evaluating Open-ended Text Generation

Danqing Wang, Kevin Yang, Hanlin Zhu, Xiaomeng Yang, Andrew Cohen, Lei Li, Yuandong Tian

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 19 pages

1. 后训练与偏好优化 4535 篇

Understanding the Logic of Direct Preference Alignment through Logic

ViLBench: A Suite for Vision-Language Process Reward Modeling

Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generation

Policy Teaching via Data Poisoning in Learning from Human Preferences

Adaptive Preference Aggregation

Generating Robot Constitutions & Benchmarks for Semantic Safety

Strong Preferences Affect the Robustness of Preference Models and Value Alignment

Learning from negative feedback, or positive feedback or both

Direct Distributional Optimization for Provable Alignment of Diffusion Models

Improving Long-Text Alignment for Text-to-Image Diffusion Models

RRM: Robust Reward Model Training Mitigates Reward Hacking

Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning

Uncertainty-Aware Step-wise Verification with Generative Reward Models

Process-Supervised Reinforcement Learning for Code Generation

Process Supervision-Guided Policy Optimization for Code Generation

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance

Aligning LLMs with Domain Invariant Reward Models

ArtAug: Enhancing Text-to-Image Generation through Synthesis-Understanding Interaction

TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback

Evaluating and Aligning CodeLLMs on Human Preference

Token-level Proximal Policy Optimization for Query Generation

Learning Goal-Conditioned Representations for Language Reward Models

Improving Reward Models with Synthetic Critiques

Preferential Normalizing Flows

Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing

Intuitions of Compromise: Utilitarianism vs. Contractualism

A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning Stick

Learning Personalized Alignment for Evaluating Open-ended Text Generation