语言大模型 / LLM - arXivDaily 专题

2511.14476 2025-11-27 cs.AI 90%

Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior

在大型语言模型对齐中操作多元价值观揭示了安全、包容性和模型行为之间的权衡

Dalia Ali, Dora Zhao, Allison Koenecke, Orestis Papakyriakopoulos

机构 * Technical University of Munich（慕尼黑技术大学）； Stanford University（斯坦福大学）； Cornell University（康奈尔大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

AI总结本研究探讨了在大型语言模型对齐中融入多元价值观的影响，发现不同群体偏好和设计选择对模型行为和安全性有显著影响。

2505.19387 2025-11-27 cs.LG cs.SY eess.SY math.OC 90%

Alignment of large language models with constrained learning

受限学习下的大语言模型对齐

Botong Zhang, Shuo Li, Ignacio Hounie, Osbert Bastani, Dongsheng Ding, Alejandro Ribeiro

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

AI总结本文提出了一种基于拉格朗日对偶性的迭代对偶对齐方法，用于在满足约束条件下优化大语言模型策略，通过实验验证了其在受限对齐问题中的有效性。

Comments 51 pages, 5 figures, 11 tables; Accepted to NeurIPS 2025

2507.05158 2025-10-30 cs.CL 90%

Steering Information Utility in Key-Value Memory for Language Model Post-Training

Chunyuan Deng, Ruidi Chang, Hanjie Chen

机构 * Dept. of Computer Science Rice University（计算机科学系瑞士大学）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；pretraining(abstract)；SFT(abstract)

Comments NeurIPS 2025

2510.12195 2025-10-15 cs.CL 90%

DPO-Tuned Large Language Models for Segmentation in Simultaneous Speech Translation

Zeyu Yang, Satoshi Nakamura

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

2502.13146 2025-09-23 cs.CV cs.LG 90%

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu

机构 * Texas A&M University（德克萨斯大学）； University of Michigan（密歇根大学）； UIUC（伊利诺伊大学香槟分校）； UNC Chapel Hill（北卡罗来纳大学教堂山分校）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；RLHF(abstract)

Comments Published at EMNLP 2025

2310.08164 2025-09-22 cs.LG 90%

Interpreting Learned Feedback Patterns in Large Language Models

Luke Marks, Amir Abdullah, Clement Neo, Rauno Arike, David Krueger, Philip Torr, Fazl Barez

机构 * Luke Marks ； Amir Abdullah ； Clement Neo ； Rauno Arike ； David Krueger ； Philip Torr ； Fazl Barez

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments 19 pages, 8 figures. Accepted to NeurIPS 2024

2506.23998 2025-08-12 cs.CL 90%

Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning

Seungjun Yi, Joakim Nguyen, Huimin Xu, Terence Lim, Andrew Well, Mia Markey, Ying Ding

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments Presented at ACL 2025 SRW

2502.18679 2025-07-25 cs.CL 90%

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data

Siqi Guo, Ilgee Hong, Vicente Balmaseda, Changlong Yu, Liang Qiu, Xin Liu, Haoming Jiang, Tuo Zhao, Tianbao Yang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；SFT(abstract)；preference optimization(abstract)

Comments 18 pages, 7 figures

2411.01834 2025-05-28 cs.CL eess.AS 90%

Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback

Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav, Yile Gu, Ankur Gandhe, Hung-yi Lee, Ivan Bulyko

机构 * Amazon AGI（亚马逊人工智能实验室）； Graduate Institute of Communication Engineering, National Taiwan University（国立台湾大学通信工程研究所）

专题命中后训练与偏好优化：language model(title,abstract)；SLM(title,abstract)；large language model(abstract)；preference optimization(abstract)

Comments Accepted by ACL 2025

2505.08507 2025-05-14 cs.LG 90%

InfoPO: On Mutual Information Maximization for Large Language Model Alignment

Teng Xiao, Zhen Ge, Sujay Sanghavi, Tian Wang, Julian Katz-Samuels, Marc Versage, Qingjun Cui, Trishul Chilimbi

机构 * Amazon（亚马逊）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；preference optimization(abstract)

Comments NAACL 2025

2505.01073 2025-05-05 cs.AI 90%

Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

Zongyuan Li, Pengfei Li, Runnan Qi, Yanan Ni, Lumin Jiang, Hui Wu, Xuebo Zhang, Kuihua Huang, Xian Guo

机构 * College of Artificial Intelligence, Nankai University（南开大学人工智能学院）； Laboratory for Big Data and Decision, National University of Defense Technology（国防科技大学大数据与决策实验室）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

2504.14838 2025-04-22 cs.AI 90%

Establishing Reliability Metrics for Reward Models in Large Language Models

Yizhou Chen, Yawen Liu, Xuesi Wang, Qingtao Yu, Guangda Huzhang, Anxiang Zeng, Han Yu, Zhiming Zhou

机构 * Nanyang Technological University（新加坡国立大学）； Shanghai University of Finance and Economics（上海金融学院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2411.15124 2025-04-16 cs.CL 90%

Tulu 3: Pushing Frontiers in Open Language Model Post-Training

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, Hannaneh Hajishirzi

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；SFT(abstract)；preference optimization(abstract)

Comments Added Tulu 3 405B results and additional analyses

2410.17509 2025-04-15 cs.LG 90%

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models

Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, Sijia Liu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments NeurIPS'24

2503.15129 2025-03-20 cs.AI 90%

Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models

Man Fai Wong, Chee Wei Tan

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2503.09407 2025-03-13 cs.CL 90%

Got Compute, but No Data: Lessons From Post-training a Finnish LLM

Elaine Zosa, Ville Komulainen, Sampo Pyysalo

专题命中后训练与偏好优化：LLM(title,abstract)；post-training(title,abstract)；instruction tuning(abstract)；preference optimization(abstract)

Comments 7 pages

Journal ref Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)

2501.15007 2025-01-28 cs.AI cs.CE q-bio.QM 90%

Controllable Protein Sequence Generation with LLM Preference Optimization

Xiangyu Liu, Yi Liu, Silei Chen, Wei Hu

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted in the 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025)

2501.01336 2025-01-03 cs.CL 90%

Aligning Large Language Models for Faithful Integrity Against Opposing Argument

Yong Zhao, Yang Deng, See-Kiong Ng, Tat-Seng Chua

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments 17 pages, 5 figures

2412.10742 2024-12-17 cs.CL 90%

WEPO: Web Element Preference Optimization for LLM-based Web Navigation

Jiarun Liu, Jia Hao, Chunhong Zhang, Zheng Hu

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments Published at AAAI 2025

2411.02712 2024-11-06 cs.CV cs.AI 90%

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Yuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen Kan

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

Comments EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

2410.14184 2024-10-21 cs.CL 90%

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time

Mozhi Zhang, Pengyu Wang, Chenkun Tan, Mianqiu Huang, Dong Zhang, Yaqian Zhou, Xipeng Qiu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)

Comments 19 pages, 6 figures

2410.06682 2024-10-14 cs.CV cs.CL eess.IV 90%

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zujun Ma, Chao Zhang

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

2408.14874 2024-08-30 cs.CL 90%

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data

Han Xia, Songyang Gao, Qiming Ge, Zhiheng Xi, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)

2407.13928 2024-07-22 cs.CL 90%

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization

Ahmed Allam

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

2312.15997 2024-07-04 cs.CL 90%

Aligning Large Language Models with Human Preferences through Representation Engineering

Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2405.20253 2024-05-31 cs.CL 90%

Evaluating Large Language Model Biases in Persona-Steered Generation

Andy Liu, Mona Diab, Daniel Fried

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments Accepted to Findings of ACL 2024. Code and data available at https://github.com/andyjliu/persona-steered-generation-bias

2404.01258 2024-04-03 cs.CV cs.AI 90%

Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward

Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, Yiming Yang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

2403.18341 2024-03-28 cs.CL 90%

IterAlign: Iterative Constitutional Alignment of Large Language Models

Xiusi Chen, Hongzhi Wen, Sreyashi Nag, Chen Luo, Qingyu Yin, Ruirui Li, Zheng Li, Wei Wang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments NAACL 2024

2310.14422 2024-02-02 cs.CL 90%

Large Language Models are biased to overestimate profoundness

Eugenio Herrera-Berg, Tomás Vergara Browne, Pablo León-Villagrá, Marc-Lluís Vives, Cristian Buc Calderon

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；prompting(abstract)

Comments 5 pages, 3 figures

Journal ref https://aclanthology.org/2023.emnlp-main.599

2312.07876 2023-12-14 cs.AI 90%

Causality Analysis for Evaluating the Security of Large Language Models

Wei Zhao, Zhe Li, Jun Sun

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

1. 后训练与偏好优化 4535 篇

Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior

Alignment of large language models with constrained learning

Steering Information Utility in Key-Value Memory for Language Model Post-Training

DPO-Tuned Large Language Models for Segmentation in Simultaneous Speech Translation

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization

Interpreting Learned Feedback Patterns in Large Language Models

Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data

Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback

InfoPO: On Mutual Information Maximization for Large Language Model Alignment

Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

Establishing Reliability Metrics for Reward Models in Large Language Models

Tulu 3: Pushing Frontiers in Open Language Model Post-Training

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models

Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models

Got Compute, but No Data: Lessons From Post-training a Finnish LLM

Controllable Protein Sequence Generation with LLM Preference Optimization

Aligning Large Language Models for Faithful Integrity Against Opposing Argument

WEPO: Web Element Preference Optimization for LLM-based Web Navigation

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization

Aligning Large Language Models with Human Preferences through Representation Engineering

Evaluating Large Language Model Biases in Persona-Steered Generation

Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward

IterAlign: Iterative Constitutional Alignment of Large Language Models

Large Language Models are biased to overestimate profoundness

Causality Analysis for Evaluating the Security of Large Language Models