语言大模型 / LLM - arXivDaily 专题

2507.02406 2025-07-04 cs.LG 85%

Improving Consistency in Vehicle Trajectory Prediction Through Preference Optimization

Caio Azevedo, Lina Achaji, Stefano Sabatini, Nicola Poerio, Grzegorz Bartyzel, Sascha Hornauer, Fabien Moutarde

机构 * Stellantis（斯特拉蒂亚斯）； École des Mines de Paris（巴黎矿学院）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Accepted for publication at ITSC 2025

2502.16825 2025-07-01 cs.CL 85%

Finding the Sweet Spot: Preference Data Construction for Scaling Preference Optimization

Yao Xiao, Hai Ye, Linyao Chen, Hwee Tou Ng, Lidong Bing, Xiaoli Li, Roy Ka-wei Lee

机构 * Singapore University of Technology and Design（新加坡科技设计大学）； National University of Singapore（国立新加坡大学）； The University of Tokyo（东京大学）； MiroMind（MiroMind公司）； Institute for Infocomm Research, A*Star, Singapore（新加坡*星研究院信息与通信研究所）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL25 Main

2506.18032 2025-06-24 cs.LG 85%

Why Do Some Language Models Fake Alignment While Others Don't?

Abhay Sheshadri, John Hughes, Julian Michael, Alex Mallen, Arun Jose, Janus, Fabien Roger

机构 * Anthropic ； Scale AI ； Redwood Research

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；post-training(abstract)；分类 cs.LG

2506.16655 2025-06-23 cs.CL 85%

Arch-Router: Aligning LLM Routing with Human Preferences

Co Tran, Salman Paracha, Adil Hafeez, Shuguang Chen

机构 * Katanemo Labs, Inc.（Katanemo实验室）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2506.03557 2025-06-05 cs.CL 85%

BPO: Revisiting Preference Modeling in Direct Preference Optimization

Lin Sun, Chuang Liu, Peng Liu, Bingyang Li, Weijia Lu, Ning Wu

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2506.00569 2025-06-03 cs.LG 85%

AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

Nicholas E. Corrado, Julian Katz-Samuels, Adithya Devraj, Hyokun Yun, Chao Zhang, Yi Xu, Yi Pan, Bing Yin, Trishul Chilimbi

机构 * University of Wisconsin–Madison（威斯康星大学麦迪逊分校）； Amazon（亚马逊）； Georgia Institute of Technology（佐治亚理工学院）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ACL 2025, Main Conference

2505.23927 2025-06-02 cs.LG 85%

Thompson Sampling in Online RLHF with General Function Approximation

Songtao Feng, Jie Fu

机构 * Department of Electrical and Computer Engineering, University of Florida（电气与计算机工程系，佛罗里达大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2505.23749 2025-05-30 cs.LG cs.GT 85%

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?

Paul Gölz, Nika Haghtalab, Kunhe Yang

机构 * Cornell University（康奈尔大学）； UC Berkeley（加州大学伯克利分校）

专题命中后训练与偏好优化：preference optimization(title)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2505.22172 2025-05-29 cs.CL 85%

Reverse Preference Optimization for Complex Instruction Following

Xiang Huang, Ting-En Lin, Feiteng Fang, Yuchuan Wu, Hangyu Li, Yuzhong Qu, Fei Huang, Yongbin Li

机构 * Tongyi Lab（通义实验室）； State Key Laboratory for Novel Software Technology（新型软件技术国家重点实验室）； Nanjing University（南京大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL 2025 Findings

2505.19100 2025-05-27 cs.CL cs.CV 85%

ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning

Yeyuan Wang, Dehong Gao, Rujiao Long, Lei Yi, Linbo Jin, Libin Yang, Xiaoyan Cai

机构 * Northwestern Polytechnical University, School of Automation, Xi’an, China（西北工业大学自动化学院，西安，中国）； Northwestern Polytechnical University, School of Cybersecurity, Xi’an, China（西北工业大学网络安全学院，西安，中国）； Binjiang Institute of Artificial Intelligence, ZJUT, Hangzhou, China（浙大宁波理工学院滨海人工智能研究院，杭州，中国）； Alibaba Group, Hangzhou, China（阿里巴巴集团，杭州，中国）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ACL 2025 findings

2505.18531 2025-05-27 cs.AI cs.CV 85%

Generative RLHF-V: Learning Principles from Multi-modal Human Preference

Jiayi Zhou, Jiaming Ji, Boyuan Chen, Jiapeng Sun, Wenqi Chen, Donghai Hong, Sirui Han, Yike Guo, Yaodong Yang

机构 * Peking University（北京大学）； Hong Kong University of Science and Technology（香港科技大学）； University College London（伦敦大学学院）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 9 pages, 8 figures

2501.18101 2025-05-23 cs.CL 85%

Diverse Preference Optimization

Jack Lanchantin, Angelica Chen, Shehzaad Dhuliawala, Ping Yu, Jason Weston, Sainbayar Sukhbaatar, Ilia Kulikov

机构 * Meta

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL

2407.06443 2025-04-29 cs.AI 85%

Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment

Qizhang Feng, Siva Rajesh Kasa, Santhosh Kumar Kasa, Hyokun Yun, Choon Hui Teo, Sravan Babu Bodapati

机构 * Amazon Inc.（亚马逊公司）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2504.15804 2025-04-23 cs.AR cs.AI 85%

Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback

Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang

机构 * City University of Hong Kong（香港城市大学）； Southeast University（东南大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2504.13122 2025-04-18 cs.CV cs.LG 85%

VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models

Haojian Huang, Haodong Chen, Shengqiong Wu, Meng Luo, Jinlan Fu, Xinya Du, Hanwang Zhang, Hao Fei

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Code and Data: https://github.com/HaroldChen19/VistaDPO

2411.08923 2025-03-27 cs.CV cs.LG 85%

Aligning Visual Contrastive learning models via Preference Optimization

Amirabbas Afzali, Borna Khodabandeh, Ali Rasekh, Mahyar JafariNodeh, Sepehr kazemi, Simon Gottschalk

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.LG

2503.18454 2025-03-25 cs.CV cs.LG 85%

InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment

Yunhong Lu, Qichao Wang, Hengyuan Cao, Xierui Wang, Xiaoyin Xu, Min Zhang

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Accepted by CVPR2025

2503.14189 2025-03-19 cs.CL cs.CV 85%

Towards Harmless Multimodal Assistants with Blind Preference Optimization

Yongqi Li, Lu Yang, Jian Wang, Runyang You, Wenjie Li, Liqiang Nie

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2503.05079 2025-03-10 cs.LG 85%

On a Connection Between Imitation Learning and RLHF

Teng Xiao, Yige Yuan, Mingxiao Li, Zhengyu Chen, Vasant G Honavar

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ICLR 2025

2410.09724 2025-03-04 cs.CL 85%

Taming Overconfidence in LLMs: Reward Calibration in RLHF

Jixuan Leng, Chengsong Huang, Banghua Zhu, Jiaxin Huang

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2410.17146 2025-03-03 cs.LG cs.CV 85%

LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging

Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, Francois Fleuret, Pascal Frossard

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；RLHF(abstract)；分类 cs.LG

Comments The first two authors contributed equally to this work. Accepted at ICLR 2025. Project website: https://lines-merging.github.io

2408.07471 2025-02-19 cs.CL 85%

Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization

Yuxin Jiang, Bo Huang, Yufei Wang, Xingshan Zeng, Liangyou Li, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Wei Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 20 pages, 9 figures, 12 tables. Accepted at ICLR 2025

2502.11520 2025-02-18 cs.CL 85%

AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification

Xiaoyu Tan, Tianchu Yao, Chao Qu, Bin Li, Minghao Yang, Dakuan Lu, Haozhe Wang, Xihe Qiu, Wei Chu, Yinghui Xu, Yuan Qi

专题命中后训练与偏好优化：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Under Review

2410.02743 2025-02-18 cs.CL 85%

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

Yekun Chai, Haoran Sun, Huang Fang, Shuohuan Wang, Yu Sun, Hua Wu

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2501.18457 2025-02-11 cs.CL 85%

CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

Yumeng Wang, Zhiyuan Fan, Qingyun Wang, May Fung, Heng Ji

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by NAACL 2025

2501.16629 2025-01-29 cs.CL cs.CV 85%

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, See-Kiong Ng

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ICLR 2025

2411.02337 2025-01-28 cs.CL 85%

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, Yuxiao Dong

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Published as a conference paper at ICLR 2025

2412.20996 2024-12-31 cs.CL 85%

Plug-and-Play Training Framework for Preference Optimization

Jingyuan Ma, Rui Li, Zheng Li, Lei Sha, Zhifang Sui

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 12 pages, 9 figures

2404.14285 2024-12-31 cs.RO cs.AI 85%

LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots

Dongge Han, Trevor McInroe, Adam Jelley, Stefano V. Albrecht, Peter Bell, Amos Storkey

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments COLING 2025

2411.15708 2024-11-26 cs.CL 85%

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

Xiaoye Qu, Daize Dong, Xuyang Hu, Tong Zhu, Weigao Sun, Yu Cheng

专题命中后训练与偏好优化：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Technical report,13 pages

1. 后训练与偏好优化 4556 篇

Improving Consistency in Vehicle Trajectory Prediction Through Preference Optimization

Finding the Sweet Spot: Preference Data Construction for Scaling Preference Optimization

Why Do Some Language Models Fake Alignment While Others Don't?

Arch-Router: Aligning LLM Routing with Human Preferences

BPO: Revisiting Preference Modeling in Direct Preference Optimization

AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

Thompson Sampling in Online RLHF with General Function Approximation

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?

Reverse Preference Optimization for Complex Instruction Following

ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning

Generative RLHF-V: Learning Principles from Multi-modal Human Preference

Diverse Preference Optimization

Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment

Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback

VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models

Aligning Visual Contrastive learning models via Preference Optimization

InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment

Towards Harmless Multimodal Assistants with Blind Preference Optimization

On a Connection Between Imitation Learning and RLHF

Taming Overconfidence in LLMs: Reward Calibration in RLHF

LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging

Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization

AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Plug-and-Play Training Framework for Preference Optimization

LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training