语言大模型 / LLM - arXivDaily 专题

2508.08039 2025-11-05 cs.SD cs.CL cs.MM eess.AS 83%

Audio-Thinker: Guiding Audio Language Model When and How to Think via Reinforcement Learning

Shu Wu, Chenxing Li, Wenfu Wang, Hao Zhang, Hualei Wang, Meng Yu, Dong Yu

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments preprint

2506.07127 2025-10-31 cs.RO cs.AI 83%

Human-assisted Robotic Policy Refinement via Action Preference Optimization

Wenke Xia, Yichu Yang, Hongtao Wu, Xiao Ma, Tao Kong, Di Hu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China, Beijing（中国人民大学北京校区人工智能学院）； Engineering Research Center of Next-Generation Intelligent Search（下一代智能搜索与推荐工程研究中心）； Beijing Key Laboratory of Research on Large Models（北京大型模型研究重点实验室）

专题命中后训练与偏好优化：preference optimization(title,abstract)；foundation model(abstract)；分类 cs.AI

Comments Accepted By NeurIPS 2025

2501.14755 2025-10-30 cs.DC cs.AI 83%

Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models

Daoyuan Chen, Yilun Huang, Xuchen Pan, Nana Jiang, Haibin Wang, Yilei Zhang, Ce Ge, Yushuo Chen, Wenhao Zhang, Zhijian Ma, Jun Huang, Wei Lin, Yaliang Li, Bolin Ding, Jingren Zhou

机构 * Alibaba Group（阿里巴巴集团）

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(abstract)；分类 cs.AI

Comments Accepted by NeurIPS 2025 (Spotlight). 43 pages, 16 figures, 4 tables

2506.01381 2025-10-29 cs.CL 83%

AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation

Yilong Lai, Jialong Wu, Zhenglin Wang, Deyu Zhou

机构 * School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University（计算机科学与工程学院、计算机网络与信息集成重点实验室、教育部、东南大学）

专题命中后训练与偏好优化：prompting(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2025

2506.04272 2025-10-14 cs.LG 83%

Understanding the Impact of Sampling Quality in Direct Preference Optimization

Kyung Rok Kim, Yumo Bai, Chonghuan Wang, Guanting Chen

机构 * University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）； University of Texas at Dallas（德克萨斯大学达拉斯分校）

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.LG

Comments Submitted to AISTATS2026

2510.04089 2025-10-07 cs.AI 83%

SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflows

Yitong Cui, Liu Liu, Baosheng Yu, Jiayan Qiu, Xikai Zhang, Likang Xiao, Yixing Liu, Quan Chen

机构 * Hangzhou International Innovation Institute and , Beihang University（杭州国际创新研究院和北航）； School of Artificial Intelligence, Beihang University（人工智能学院，北航）； Nanyang Technological University（南洋理工大学）； University of Leicester（莱斯特大学）； China Mobile Communications Company Limited Research Institute（中国移动通信有限公司研究院）

专题命中后训练与偏好优化：preference optimization(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2507.07510 2025-10-07 cs.CV cs.LG 83%

Divergence Minimization Preference Optimization for Diffusion Model Alignment

Binxu Li, Minkai Xu, Jiaqi Han, Meihua Dang, Stefano Ermon

机构 * Stanford University（斯坦福大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.LG

2509.20384 2025-09-26 cs.CR cs.AI cs.PL cs.SE 83%

R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning

Jiayi Lin, Liangcai Su, Junzhe Li, Chenxiong Qian

机构 * The University of Hong Kong（香港大学）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)；分类 cs.AI

2509.16399 2025-09-23 cs.AI 83%

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Guojun Xiong, Milind Tambe

机构 * Department of Computer Science, Harvard University（计算机科学系，哈佛大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 28pages, 19figures

2509.09629 2025-09-12 cs.CL 83%

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen

机构 * Shandong University（山东大学）； Leiden University（莱顿大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

2508.08550 2025-08-13 cs.SD cs.CL 83%

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Chaoqun Cui, Liangbin Huang, Shijing Wang, Zhe Tong, Zhaolong Huang, Xiao Zeng, Xiaofeng Liu

机构 * Alibaba Digital Media and Entertainment Group（阿里巴巴数字媒体与娱乐集团）； School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）； Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University（北京交通大学交通数据挖掘与具身智能重点实验室）

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；分类 cs.CL

Comments This paper is accepted by ACL2025 (Main)

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025: 4524-4546

2507.11554 2025-08-05 cs.CV cs.AI 83%

Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models

Zejian Li, Yize Li, Chenye Meng, Zhongni Liu, Yang Ling, Shengyuan Zhang, Guang Yang, Changyuan Yang, Zhiyuan Yang, Lingyun Sun

机构 * Zhejiang University（浙江大学）； University of Electronic Science and Technology of China（电子科技大学）； Peking University（北京大学）； Alibaba Group（阿里巴巴集团）

专题命中后训练与偏好优化：post-training(title,abstract)；preference optimization(abstract)；分类 cs.AI

Comments Accepted by ACM MM25

2507.07939 2025-07-23 cs.CL 83%

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Guoxin Zang, Xue Li, Donglin Di, Lanshun Nie, Dechen Zhan, Yang Song, Lei Fan

机构 * Harbin Institute of Technology（哈尔滨工业大学）； University of New South Wales（新南威尔士大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by ACMMM2025

2507.09016 2025-07-17 cs.LG 83%

Enhancing RLHF with Human Gaze Modeling

Karim Galliamov, Ivan Titov, Ilya Pershin

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.LG

2507.10923 2025-07-16 cs.AI 83%

Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization

Yuhao Wang, Keyan Ding, Kehua Feng, Zeyuan Wang, Ming Qin, Xiaotong Li, Qiang Zhang, Huajun Chen

机构 * Zhejiang University（浙江大学）； ZJU-Hangzhou Global Scientific and Technological Innovation Center（浙大杭州国际科创中心）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.AI

Comments Accepted at ACL 2025 (Main Conference)

2507.05911 2025-07-09 cs.SD cs.AI eess.AS 83%

Differentiable Reward Optimization for LLM based TTS system

Changfeng Gao, Zhihao Du, Shiliang Zhang

机构 * Speech Team, Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中后训练与偏好优化：LLM(title)；language model(abstract)；RLHF(abstract)；分类 cs.AI

2506.21497 2025-06-27 cs.CL 83%

Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

Jiashuo Wang, Kaitao Song, Chunpu Xu, Changhe Song, Yang Xiao, Dongsheng Li, Lili Qiu, Wenjie Li

机构 * Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）； Microsoft Research Asia（微软亚洲研究院）

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(abstract)；分类 cs.CL

2411.04291 2025-06-23 cs.CL cs.CV 83%

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

Saketh Bachu, Erfan Shayegani, Rohit Lal, Trishna Chakraborty, Arindam Dutta, Chengyu Song, Yue Dong, Nael Abu-Ghazaleh, Amit K. Roy-Chowdhury

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments Accepted by ICML 2025 as a spotlight poster

2502.07599 2025-06-09 cs.CL 83%

DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Xiliang Yang, Feng Jiang, Qianen Zhang, Lei Zhao, Xiao Li

机构 * David S. Hippocampus Department of Computer Science（计算机科学系）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2506.02708 2025-06-04 cs.CV cs.CL 83%

Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation

Naoto Tanji, Toshihiko Yamasaki

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted to ICIP2025

2506.02211 2025-06-04 cs.AI 83%

Improving LLM-Generated Code Quality with GRPO

Maxime Robeyns, Laurence Aitchison

机构 * University of Bristol（布里斯托大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2412.17018 2025-06-04 cs.AI 83%

GAS: Generative Auto-bidding with Post-training Search

Yewen Li, Shuai Mao, Jingtong Gao, Nan Jiang, Yunjian Xu, Qingpeng Cai, Fei Pan, Peng Jiang, Bo An

机构 * Nanyang Technological University（南洋理工大学）； The Chinese University of Hong Kong（香港中文大学）； City University of Hong Kong（香港城市大学）； Kuaishou Technology（快手科技）

专题命中后训练与偏好优化：post-training(title,abstract)；foundation model(abstract)；分类 cs.AI

2505.19095 2025-05-27 cs.AI 83%

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang

机构 * School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2505.15948 2025-05-23 cs.CL cs.DL cs.SI 83%

Citation Parsing and Analysis with Language Models

Parth Sarin, Juan Pablo Alperin

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)；分类 cs.CL

Comments Presented at the Workshop on Open Citations & Open Scholarly Metadata 2025

2412.20299 2025-05-14 cs.CL 83%

No Preference Left Behind: Group Distributional Preference Optimization

Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, Junjie Hu

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校）； Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）； Stanford University（斯坦福大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2505.06548 2025-05-13 cs.CL 83%

REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback

Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal

机构 * Indian Institute of Technology, Kharagpur（印度理工学院，克哈格普尔）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 11 pages

2411.03700 2025-05-13 cs.CL 83%

The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models

Anaelia Ovalle, Krunoslav Lehman Pavasovic, Louis Martin, Luke Zettlemoyer, Eric Michael Smith, Kai-Wei Chang, Adina Williams, Levent Sagun

机构 * FAIR, Meta（FAIR与Meta）； Meta United States（Meta美国分公司）； Meta France（Meta法国分公司）； Gen AI, Meta（Meta生成式人工智能部门）； UCLA United States（加州大学洛杉矶分校）

专题命中后训练与偏好优化：language model(title)；SFT(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted to 2025 ACM FAccT

2505.02363 2025-05-06 cs.CL 83%

SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning

Tianjian Li, Daniel Khashabi

机构 * Johns Hopkins University（约翰霍普金斯大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments To appear in ICML 2025

2504.06562 2025-04-18 cs.CL 83%

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion

Longguang Zhong, Fanqi Wan, Ziyi Yang, Guosheng Liang, Tianyuan Shi, Xiaojun Quan

专题命中后训练与偏好优化：preference optimization(title,abstract)；SFT(abstract)；分类 cs.CL

2410.09893 2025-04-07 cs.CL 83%

RMB: Comprehensively Benchmarking Reward Models in LLM Alignment

Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ICLR2025

1. 后训练与偏好优化 4556 篇

Audio-Thinker: Guiding Audio Language Model When and How to Think via Reinforcement Learning

Human-assisted Robotic Policy Refinement via Action Preference Optimization

Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models

AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation

Understanding the Impact of Sampling Quality in Direct Preference Optimization

SPOGW: a Score-based Preference Optimization method via Group-Wise comparison for workflows

Divergence Minimization Preference Optimization for Diffusion Model Alignment

R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Enhancing RLHF with Human Gaze Modeling

Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization

Differentiable Reward Optimization for LLM based TTS system

Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation

Improving LLM-Generated Code Quality with GRPO

GAS: Generative Auto-bidding with Post-training Search

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Citation Parsing and Analysis with Language Models

No Preference Left Behind: Group Distributional Preference Optimization

REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback

The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models

SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion

RMB: Comprehensively Benchmarking Reward Models in LLM Alignment