语言大模型 / LLM - arXivDaily 专题

2502.02950 2025-12-29 eess.AS cs.SD 82%

Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech

细粒度偏好优化提升零样本文本到语音

Jixun Yao, Yuguang Yang, Yu Pan, Yuan Feng, Ziqian Ning, Jianhao Ye, Hongbin Zhou, Lei Xie

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)

AI总结细粒度偏好优化方法通过解决生成样本中的局部问题，提升零样本文本到语音系统的鲁棒性和可懂度。

Comments Accepted By IEEE TASLP

2512.15644 2025-12-18 cs.CV 82%

InpaintDPO: Mitigating Spatial Relationship Hallucinations in Foreground-conditioned Inpainting via Diverse Preference Optimization

InpaintDPO：通过多样偏好优化缓解前景条件修复中的空间关系幻觉

Qirui Li, Yizhe Tang, Ran Yi, Guangben Lu, Fangyuan Zou, Peng Shu, Huan Yu, Jie Jiang

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)

AI总结 InpaintDPO通过多样偏好优化解决前景条件修复中的空间关系幻觉问题，提升背景与前景间的一致性与合理性。

2512.10264 2025-12-16 cs.SD 82%

MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generation

MR-FlowDPO：多奖励直接偏好优化用于流匹配文本到音乐生成

Alon Ziv, Sanyuan Chen, Andros Tjandra, Yossi Adi, Wei-Ning Hsu, Bowen Shi

机构 * FAIR Team, Meta MSL（Meta MSL FAIR团队）； The Hebrew University of Jerusalem（耶路撒冷希伯来大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；prompting(abstract)

AI总结 MR-FlowDPO通过多奖励直接偏好优化提升文本到音乐生成的质量，增强音频质量、文本对齐和音乐性。

2511.15661 2025-11-21 cs.CV cs.AI cs.CL cs.LG 82%

VisPlay: Self-Evolving Vision-Language Models from Images

VisPlay: 从图像中自我进化视觉-语言模型

Yicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang, Yonghui Yang

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Washington University in St. Louis（华盛顿大学圣路易斯分校）； University of Maryland（马里兰大学）； National University of Singapore（新加坡国立大学）

专题命中后训练与偏好优化：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 VisPlay通过自我进化强化学习框架，利用未标注图像数据提升视觉-语言模型的推理能力，实现多模态智能的可扩展发展。

2506.14157 2025-11-18 cs.CL cs.AI cs.LG 82%

DCRM: A Heuristic to Measure Response Pair Quality in Preference Optimization

Chengyu Huang, Tanya Goyal

机构 * Cornell University（康奈尔大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2412.17417 2025-11-13 cs.CV 82%

Synth-Align: Improving Trustworthiness in Vision-Language Model with Synthetic Preference Data Alignment

Robert Wijaya, Ngoc-Bao Nguyen, Ngai-Man Cheung

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)

2506.11234 2025-11-07 cs.RO cs.CV 82%

Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving

Luke Rowe, Rodrigue de Schaetzen, Roger Girgis, Christopher Pal, Liam Paull

机构 * Mila - Quebec AI Institute（魁北克人工智能研究所）； Université de Montréal（蒙特利尔大学）； Polytechnique Montréal（蒙特利尔理工学院）； CIFAR AI Chair（CIFAR人工智能主席）

专题命中后训练与偏好优化：post-training(title)；language model(abstract)；pretraining(abstract)

2412.16378 2025-11-06 cs.LG cs.AI cs.CL 82%

REFA: Reference Free Alignment for multi-preference optimization

Taneesh Gupta, Rahul Madhavan, Xuchao Zhang, Chetan Bansal, Saravan Rajmohan

机构 * MicrosoftIndian Institute of Science (IISc), Bangalore（微软印度科学研究院（IISc），班加罗尔）

专题命中后训练与偏好优化：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2504.06020 2025-10-27 cs.AI cs.CL cs.LG 82%

Information-Theoretic Reward Decomposition for Generalizable RLHF

Liyuan Mao, Haoran Xu, Amy Zhang, Weinan Zhang, Chenjia Bai

机构 * Shanghai Jiao Tong University（上海交通大学）； UT Austin（得克萨斯大学奥斯汀分校）； Institute of Artificial Intelligence, China Telecom（中国电信人工智能研究院）

专题命中后训练与偏好优化：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work done during internships at Institute of Artificial Intelligence (TeleAI), China Telecom

2506.11430 2025-10-21 cs.CV 82%

Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization

Jingfeng Guo, Jian Liu, Jinnan Chen, Shiwei Mao, Changrong Hu, Puhua Jiang, Junlin Yu, Jing Xu, Qi Liu, Lixin Xu, Zhuo Chen, Chunchao Guo

机构 * South China University of Technology（南方科技大学）； Hong Kong University of Science and Technology（香港科技大学）； National University of Singapore（国立新加坡大学）； Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）； University of Science and Technology of China（中国科学技术大学）； Beijing Normal University（北京师范大学）； Tencent Hunyuan（腾讯文盈）

专题命中后训练与偏好优化：preference optimization(title,abstract)；post-training(abstract)

2510.05095 2025-10-07 cs.LG cs.AI cs.CL 82%

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models

Mingkang Zhu, Xi Chen, Bei Yu, Hengshuang Zhao, Jiaya Jia

机构 * The Chinese University of Hong Kong（香港中文大学）； The University of Hong Kong（香港大学）； The Hong Kong University of Science and Technology（香港科学与技术大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2502.01051 2025-10-03 cs.CV 82%

Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization

Tao Zhang, Cheng Da, Kun Ding, Huan Yang, Kun Jin, Yan Li, Tingting Gao, Di Zhang, Shiming Xiang, Chunhong Pan

机构 * MAIS, CASIA（中国科学院自动化研究所MAIS实验室）； Kuaishou Technology（快手科技）； School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)

Comments NeurIPS 2025

2509.24418 2025-09-30 cs.CR 82%

GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasoners

Haoran Li, Yulin Chen, Jingru Zeng, Hao Peng, Huihao Jing, Wenbin Hu, Xi Yang, Ziqian Zeng, Sirui Han, Yangqiu Song

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)

2412.14487 2025-09-24 cs.CV 82%

Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation

Jihao Gu, Yingyao Wang, Meng Cao, Pi Bu, Jun Song, Yancheng He, Shilong Li, Bo Zheng

机构 * Taobao & Tmall Group of Alibaba（淘宝与天猫集团）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)

2505.03233 2025-08-28 cs.RO 82%

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang, Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Wenhao Zhang, Heming Cui, Zhizheng Zhang, He Wang

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(abstract)

2409.14836 2025-08-26 cs.CL cs.AI cs.LG 82%

Orthogonal Finetuning for Direct Preference Optimization

Chenxu Yang, Ruipeng Jia, Naibin Gu, Zheng Lin, Siyuan Chen, Chao Pang, Weichong Yin, Yu Sun, Hua Wu, Weiping Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2506.11144 2025-06-16 cs.CV 82%

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation

Chao Liang, Jianwen Jiang, Wang Liao, Jiaqi Yang, Zerong zheng, Weihong Zeng, Han Liang

机构 * ByteDance（字节跳动）

专题命中后训练与偏好优化：preference optimization(title,abstract)；post-training(abstract)

Comments Homepage: https://alignhuman.github.io/

2501.08617 2025-06-11 cs.LG cs.AI cs.CL 82%

RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation

Kaiqu Liang, Haimin Hu, Ryan Liu, Thomas L. Griffiths, Jaime Fernández Fisac

机构 * Princeton University（普林斯顿大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 18 figures

2505.07271 2025-05-13 cs.CL cs.AI cs.LG 82%

On the Robustness of Reward Models for Language Model Alignment

Jiwoo Hong, Noah Lee, Eunki Kim, Guijin Son, Woojin Chung, Aman Gupta, Shao Tang, James Thorne

专题命中后训练与偏好优化：language model(title)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025

2503.13690 2025-05-09 cs.CL cs.AI cs.LG 82%

Atyaephyra at SemEval-2025 Task 4: Low-Rank Negative Preference Optimization

Jan Bronec, Jindřich Helcl

机构 * Charles University, Faculty of Mathematics and Physics（查理大学数学与物理学院）

专题命中后训练与偏好优化：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 3 figures, accepted to SemEval workshop proceedings at ACL 2025

2504.14534 2025-04-22 cs.CV 82%

SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization

Liang Peng, Boxi Wu, Haoran Cheng, Yibo Zhao, Xiaofei He

机构 * FABU Inc.（FABU公司）； Zhejiang University（浙江大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；SFT(abstract)

2503.14504 2025-03-25 cs.CV 82%

Aligning Multimodal LLM with Human Preference: A Survey

Tao Yu, Yi-Fan Zhang, Chaoyou Fu, Junkang Wu, Jinda Lu, Kun Wang, Xingyu Lu, Yunhang Shen, Guibin Zhang, Dingjie Song, Yibo Yan, Tianlong Xu, Qingsong Wen, Zhang Zhang, Yan Huang, Liang Wang, Tieniu Tan

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)

Comments Project page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment

2311.12372 2025-03-24 cs.CR 82%

PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network

Ruitong Liu, Yanbin Wang, Haitao Xu, Zhan Qin, Fan Zhang, Yiwei Liu, Zheng Cao

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)

Comments 18 pages, 8 figures

2501.09695 2025-03-04 cs.CV 82%

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, Dongsheng Li

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)

Comments Accepted by CVPR 2025

2501.00560 2025-02-12 cs.CL cs.AI cs.LG 82%

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference

Mingqi Gao, Yixin Liu, Xinyu Hu, Xiaojun Wan, Jonathan Bragg, Arman Cohan

专题命中后训练与偏好优化：LLM(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Findings of NAACL 2025

2403.13352 2025-01-03 cs.CV 82%

AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation

Jingkun An, Yinghao Zhu, Zongjian Li, Enshen Zhou, Haoran Feng, Xijie Huang, Bohua Chen, Yemin Shi, Chengwei Pan

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)

Comments Accepted by AAAI-2025

2407.14916 2024-11-07 cs.CL cs.AI cs.LG 82%

Improving Context-Aware Preference Modeling for Language Models

Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni

专题命中后训练与偏好优化：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024. 10 pages (29 with references and appendix)

2409.10157 2024-09-17 eess.AS cs.SD eess.SP 82%

Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization

Xiaoxue Gao, Chen Zhang, Yiming Chen, Huayun Zhang, Nancy F. Chen

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)

Comments 5 pages

2408.16500 2024-08-30 cs.CV 82%

CogVLM2: Visual Language Models for Image and Video Understanding

Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, Lei Zhao, Zhuoyi Yang, Xiaotao Gu, Xiaohan Zhang, Guanyu Feng, Da Yin, Zihan Wang, Ji Qi, Xixuan Song, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Yuxiao Dong, Jie Tang

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)

2408.10433 2024-08-21 cs.CV 82%

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

Yassine Ouali, Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)

Comments Accepted at ECCV 2024

1. 后训练与偏好优化 4556 篇

Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech

InpaintDPO: Mitigating Spatial Relationship Hallucinations in Foreground-conditioned Inpainting via Diverse Preference Optimization

MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generation

VisPlay: Self-Evolving Vision-Language Models from Images

DCRM: A Heuristic to Measure Response Pair Quality in Preference Optimization

Synth-Align: Improving Trustworthiness in Vision-Language Model with Synthetic Preference Data Alignment

Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving

REFA: Reference Free Alignment for multi-preference optimization

Information-Theoretic Reward Decomposition for Generalizable RLHF

Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models

Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization

GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasoners

Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Orthogonal Finetuning for Direct Preference Optimization

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation

RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation

On the Robustness of Reward Models for Language Model Alignment

Atyaephyra at SemEval-2025 Task 4: Low-Rank Negative Preference Optimization

SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization

Aligning Multimodal LLM with Human Preference: A Survey

PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference

AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation

Improving Context-Aware Preference Modeling for Language Models

Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization

CogVLM2: Visual Language Models for Image and Video Understanding

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs