语言大模型 / LLM - arXivDaily 专题

2505.17589 2025-05-28 cs.SD cs.AI eess.AS 87%

CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training

Zhihao Du, Changfeng Gao, Yuxuan Wang, Fan Yu, Tianyu Zhao, Hao Wang, Xiang Lv, Hui Wang, Chongjia Ni, Xian Shi, Keyu An, Guanrou Yang, Yabin Li, Yanni Chen, Zhifu Gao, Qian Chen, Yue Gu, Mengzhe Chen, Yafeng Chen, Shiliang Zhang, Wen Wang, Jieping Ye

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Preprint, work in progress

2410.15610 2025-05-27 cs.LG 87%

On The Global Convergence Of Online RLHF With Neural Parametrization

Mudit Gaur, Amrit Singh Bedi, Raghu Pasupathy, Vaneet Aggarwal

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Comments The updated version of this paper is arXiv:2503.17644

2505.15045 2025-05-22 cs.CL 87%

Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective

Siyue Zhang, Yilun Zhao, Liyuan Geng, Arman Cohan, Anh Tuan Luu, Chen Zhao

机构 * Nanyang Technological University（南洋理工大学）； Yale University（耶鲁大学）； NYU Shanghai（纽约大学上海校区）； Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-国立科技大学新加坡联合研究机构）； Center for Data Science New York University（纽约大学数据科学中心）

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；post-training(abstract)

2505.12845 2025-05-20 cs.AI 87%

Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks

Ruopei Sun, Jianfeng Cai, Jinhua Zhu, Kangwen Zhao, Dongyun Xue, Wengang Zhou, Li Li, Houqiang Li

机构 * University of Science and Technology of China（中国科学技术大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2410.04350 2025-04-16 cs.CL 87%

TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights

Aiwei Liu, Haoping Bai, Zhiyun Lu, Yanchao Sun, Xiang Kong, Simon Wang, Jiulong Shan, Albin Madappally Jose, Xiaojiang Liu, Lijie Wen, Philip S. Yu, Meng Cao

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Published in ICLR 2025, code in https://github.com/exlaw/TIS-DPO

2504.05309 2025-04-09 cs.IR cs.AI 87%

IterQR: An Iterative Framework for LLM-based Query Rewrite in e-Commercial Search System

Shangyu Chen, Xinyu Jia, Yingfei Zhang, Shuai Zhang, Xiang Li, Wei Lin

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

2503.18013 2025-03-25 cs.CV cs.AI 87%

Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning

Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, Jinqiao Wang

专题命中后训练与偏好优化：language model(title,abstract)；pretraining(abstract)；post-training(abstract)；preference optimization(abstract)

Comments Project in development. Github: https://github.com/jefferyZhan/Griffon/tree/master/Vision-R1

2502.16182 2025-03-21 cs.CL 87%

IPO: Your Language Model is Secretly a Preference Classifier

Shivank Garg, Ayush Singh, Shweta Singh, Paras Chopra

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2503.11701 2025-03-18 cs.LG 87%

A Survey of Direct Preference Optimization

Shunyu Liu, Wenkai Fang, Zetian Hu, Junjie Zhang, Yang Zhou, Kongcheng Zhang, Rongcheng Tu, Ting-En Lin, Fei Huang, Mingli Song, Yongbin Li, Dacheng Tao

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2503.09025 2025-03-13 cs.CL 87%

Aligning to What? Limits to RLHF Based Alignment

Logan Barnhart, Reza Akbarian Bafghi, Stephen Becker, Maziar Raissi

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

2503.06358 2025-03-11 cs.LG 87%

Language Model Personalization via Reward Factorization

Idan Shenfeld, Felix Faltings, Pulkit Agrawal, Aldo Pacchiano

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；RLHF(abstract)

2503.01076 2025-03-04 cs.LG stat.ML 87%

Active Learning for Direct Preference Optimization

Branislav Kveton, Xintong Li, Julian McAuley, Ryan Rossi, Jingbo Shang, Junda Wu, Tong Yu

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2502.14340 2025-02-21 cs.CL 87%

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective

Ruichen Shao, Bei Li, Gangao Liu, Yang Chen, Xiang Zhou, Jingang Wang, Xunliang Cai, Peng Li

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Accepted by ICLR 2025

2405.16681 2025-02-19 cs.CL 87%

Triple Preference Optimization: Achieving Better Alignment using a Single Step Optimization

Amir Saeidi, Shivanshu Verma, Aswin RRV, Kashif Rasul, Chitta Baral

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2502.08922 2025-02-14 cs.AI 87%

Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models

Xin Zhou, Yiwen Guo, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；preference optimization(abstract)

2412.04193 2025-01-07 cs.CL 87%

AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic

Nathaniel R. Robinson, Shahd Abdelmoneim, Kelly Marchisio, Sebastian Ruder

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Pre-print

2412.15623 2024-12-23 cs.CR cs.AI 87%

JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

Hongyi Li, Jiawei Ye, Jie Wu, Tianjie Yan, Chu Wang, Zhixin Li

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by AAAI 2025

2412.10616 2024-12-17 cs.LG 87%

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Avinandan Bose, Zhihan Xiong, Aadirupa Saha, Simon Shaolei Du, Maryam Fazel

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2406.10957 2024-12-10 cs.CL 87%

Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence

Junru Lu, Jiazheng Li, Siyu An, Meng Zhao, Yulan He, Di Yin, Xing Sun

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments EMNLP 2024 Main, Final Version

2411.08733 2024-11-15 cs.CL 87%

Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models

Somanshu Singla, Zhen Wang, Tianyang Liu, Abdullah Ashfaq, Zhiting Hu, Eric P. Xing

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；SFT(abstract)；RLHF(abstract)

Comments EMNLP 2024 Main

2411.01245 2024-11-05 cs.CL 87%

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment

Dongxu Liu, Bing Xu, Yinzhuo Chen, Bufan Xu, Wenpeng Lu, Muyun Yang, Tiejun Zhao

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

2410.20869 2024-10-29 cs.CL 87%

Reward Modeling with Weak Supervision for Language Models

Ben Hauptvogel, Malte Ostendorff, Georg Rehm, Sebastian Möller

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；RLHF(abstract)

2406.17453 2024-10-18 cs.CL 87%

Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain

Davide Mazzaccara, Alberto Testoni, Raffaella Bernardi

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to EMNLP 2024 (Findings)

2404.10719 2024-10-11 cs.CL 87%

Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study

Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, Yi Wu

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments 16 pages, 2 figures, 14 tables

Journal ref ICML 2024

2310.00898 2024-09-16 cs.CL 87%

Enabling Language Models to Implicitly Learn Self-Improvement

Ziqi Wang, Le Hou, Tianjian Lu, Yuexin Wu, Yunxuan Li, Hongkun Yu, Heng Ji

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；prompting(abstract)

Comments Accepted at ICLR 2024. 28 pages, 5 figures, 4 tables

2408.02923 2024-08-07 cs.CL 87%

Intermediate direct preference optimization

Atsushi Kojima

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

2407.09072 2024-07-15 cs.CL 87%

New Desiderata for Direct Preference Optimization

Xiangkun Hu, Tong He, David Wipf

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2402.13210 2024-07-04 cs.LG 87%

Bayesian Reward Models for LLM Alignment

Adam X. Yang, Maxime Robeyns, Thomas Coste, Zhengyan Shi, Jun Wang, Haitham Bou-Ammar, Laurence Aitchison

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2406.03363 2024-06-06 cs.CL 87%

LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback

Timon Ziegenbein, Gabriella Skitalinskaya, Alireza Bayat Makou, Henning Wachsmuth

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2404.09220 2024-04-16 cs.CL 87%

Compass: Large Multilingual Language Model for South-east Asia

Sophia Maria

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；SFT(abstract)；preference optimization(abstract)

1. 后训练与偏好优化 4541 篇

CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training

On The Global Convergence Of Online RLHF With Neural Parametrization

Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective

Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks

TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights

IterQR: An Iterative Framework for LLM-based Query Rewrite in e-Commercial Search System

Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning

IPO: Your Language Model is Secretly a Preference Classifier

A Survey of Direct Preference Optimization

Aligning to What? Limits to RLHF Based Alignment

Language Model Personalization via Reward Factorization

Active Learning for Direct Preference Optimization

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective

Triple Preference Optimization: Achieving Better Alignment using a Single Step Optimization

Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models

AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic

JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence

Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment

Reward Modeling with Weak Supervision for Language Models

Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain

Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study

Enabling Language Models to Implicitly Learn Self-Improvement

Intermediate direct preference optimization

New Desiderata for Direct Preference Optimization

Bayesian Reward Models for LLM Alignment

LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback

Compass: Large Multilingual Language Model for South-east Asia