语言大模型 / LLM - arXivDaily 专题

2502.13146 2025-09-23 cs.CV cs.LG 90%

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu

机构 * Texas A&M University（德克萨斯大学）； University of Michigan（密歇根大学）； UIUC（伊利诺伊大学香槟分校）； UNC Chapel Hill（北卡罗来纳大学教堂山分校）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；RLHF(abstract)

Comments Published at EMNLP 2025

2506.04039 2025-09-23 cs.CV cs.AI cs.CL 90%

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Jiulong Wu, Zhengliang Shi, Shuaiqiang Wang, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao, Min Zhang

机构 * Soochow University（苏州大学）； Baidu Inc.（百度公司）； Shandong University（山东大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments This paper is accepted by EMNLP2025

2509.16475 2025-09-23 cs.LG cs.CL 88%

Towards Universal Debiasing for Language Models-based Tabular Data Generation

Tianchun Li, Tianci Liu, Xingchen Wang, Rongzhe Wei, Pan Li, Lu Su, Jing Gao

机构 * Purdue University（普渡大学）； Georgia Institute of Technology（佐治亚理工学院）

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；preference optimization(abstract)

Comments EMNLP 2025 Findings

2509.16990 2025-09-23 cs.CL cs.AI cs.LG cs.SD eess.AS 87%

Advancing Speech Understanding in Speech-Aware Language Models with GRPO

Avishai Elmakies, Hagai Aronowitz, Nimrod Shabtay, Eli Schwartz, Ron Hoory, Avihu Dekel

机构 * IBM Research（IBM研究院）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；SFT(abstract)；分类 cs.CL、cs.AI、cs.LG

2509.16560 2025-09-23 cs.CV 85%

Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization

Ji Soo Lee, Byungoh Ko, Jaewon Cho, Howoong Lee, Jaewoon Byun, Hyunwoo J. Kim

机构 * Korea University（韩国大学）； Hanwha Vision（翰威英航）； KAIST（韩国科学技术院）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments EMNLP 2025 Findings

2509.16399 2025-09-23 cs.AI 83%

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Guojun Xiong, Milind Tambe

机构 * Department of Computer Science, Harvard University（计算机科学系，哈佛大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 28pages, 19figures

2409.17407 2025-09-23 cs.AI cs.CL 82%

Post-hoc Reward Calibration: A Case Study on Length Bias

Zeyu Huang, Zihan Qiu, Zili Wang, Edoardo M. Ponti, Ivan Titov

机构 * University of Edinburgh（爱丁堡大学）； Alibaba Group（阿里巴巴集团）； INF Technology（INF技术）； University of Amsterdam（阿姆斯特丹大学）

专题命中后训练与偏好优化：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments ICLR 2025

2504.09696 2025-09-23 cs.CL 74%

GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models

Jixiao Zhang, Chunsheng Zuo

机构 * Johns Hopkins University（约翰霍普金斯大学）

专题命中后训练与偏好优化：language model(title)；分类 cs.CL

Comments Accepted to EMNLP 2025 (Main)

2509.00707 2025-09-23 cs.CL cs.AI 73%

Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs

Daehoon Gwak, Minseo Jung, Junwoo Park, Minho Park, ChaeHun Park, Junha Hyung, Jaegul Choo

机构 * KAIST AI（韩国科学技术院人工智能研究所）； Applied Artificial Intelligence, Sungkyunkwan University（成均馆大学应用人工智能系）

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025 Main Paper (Long)

2509.17105 2025-09-23 cs.LG 70%

GRPOformer: Advancing Hyperparameter Optimization via Group Relative Policy Optimization

Haoxin Guo, Jiawen Pan, Weixin Zhai

机构 * 1College of Information ； Electrical Engineering, \ Agricultural University, Beijing 100083, China ； Electrical Engineering, \ Agricultural University, Beijing 100083, China 2Key Laboratory of Agricultural Machinery Monitoring

专题命中后训练与偏好优化：large language model(abstract)；language model(abstract)；分类 cs.LG

2509.16965 2025-09-23 cs.CL 70%

Preference Distillation via Value based Reinforcement Learning

Minchan Kwon, Junwon Ko, Kangil Kim, Junmo Kim

机构 * Korea Advanced Institute of Science and Technology (KAIST)（韩国先进科学技术研究院）； Gwangju Institute of Science and Technology (GIST)（全州科学技术院）

专题命中后训练与偏好优化：language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments 20 page

Journal ref NIPS 2025 Poster

2509.16246 2025-09-23 cs.PL cs.AR 67%

VerilogMonkey: Exploring Parallel Scaling for Automated Verilog Code Generation with LLMs

Juxin Niu, Yuxin Du, Dan Niu, Xi Wang, Zhe Jiang, Nan Guan

专题命中后训练与偏好优化：LLM(abstract)；post-training(abstract)

2505.13388 2025-09-23 cs.CL cs.AI cs.LG 67%

R3: Robust Rubric-Agnostic Reward Models

David Anugraha, Zilu Tang, Lester James V. Miranda, Hanyang Zhao, Mohammad Rifqi Farhansyah, Garry Kuwanto, Derry Wijaya, Genta Indra Winata

机构 * Stanford University（斯坦福大学）； Boston University（波士顿大学）； Columbia University（哥伦比亚大学）； University of Toronto（多伦多大学）； Institut Teknologi Bandung（Bandung 工程技术大学）； Monash University Indonesia（墨尔本大学印尼分校）； Capital One

专题命中后训练与偏好优化：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Preprint

2509.17917 2025-09-23 cs.AI 57%

Orcust: Stepwise-Feedback Reinforcement Learning for GUI Agent

Junyu Lu, Songxin Zhang, Zejian Xie, Zhuoyang Song, Jiaxing Zhang

机构 * Lionrock AI Lab（狮岩人工智能实验室）； China Merchants Research Institute of Advanced Technology（中国商人先进科技研究院）

专题命中后训练与偏好优化：LLM(abstract)；分类 cs.AI

2407.04615 2025-09-23 cs.CL 57%

On the Low-Rank Parametrization of Reward Models for Controlled Language Generation

Sergey Troshin, Vlad Niculae, Antske Fokkens

专题命中后训练与偏好优化：language model(abstract)；分类 cs.CL

Comments TMLR 2025

2509.17940 2025-09-23 cs.RO cs.CV 50%

DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving

Shuyao Shang, Yuntao Chen, Yuqi Wang, Yingyan Li, Zhaoxiang Zhang

机构 * NLPR, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所神经网络与模式识别实验室）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中后训练与偏好优化：preference optimization(abstract)

Comments NeurIPS 2025

2509.17627 2025-09-23 cs.CV 50%

OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Models

Jinshu Chen, Xinghui Li, Xu Bai, Tianxiang Ma, Pengze Zhang, Zhuowei Chen, Gen Li, Lijie Liu, Songtao Zhao, Bingchuan Li, Qian He

机构 * Intelligent Creation Lab, ByteDance（字节跳动智能创作实验室）

专题命中后训练与偏好优化：preference optimization(abstract)

Comments Github Page: https://phantom-video.github.io/OmniInsert/

1. 后训练与偏好优化 17 篇

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Towards Universal Debiasing for Language Models-based Tabular Data Generation

Advancing Speech Understanding in Speech-Aware Language Models with GRPO

Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Post-hoc Reward Calibration: A Case Study on Length Bias

GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models

Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs

GRPOformer: Advancing Hyperparameter Optimization via Group Relative Policy Optimization

Preference Distillation via Value based Reinforcement Learning

VerilogMonkey: Exploring Parallel Scaling for Automated Verilog Code Generation with LLMs

R3: Robust Rubric-Agnostic Reward Models

Orcust: Stepwise-Feedback Reinforcement Learning for GUI Agent

On the Low-Rank Parametrization of Reward Models for Controlled Language Generation

DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving

OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Models