语言大模型 / LLM - arXivDaily 专题

2601.04670 2026-01-09 cs.LG 88%

Learning Dynamics in RL Post-Training for Language Models

在语言模型中学习动态的强化学习后训练

Akiyoshi Tomihari

机构 * Department of Computer Science, The University of Tokyo（东京大学计算机科学系）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；分类 cs.LG

AI总结本文提出分类器优先强化学习（CF-RL），通过优先更新分类器来提升语言模型的对齐性和推理能力，揭示了RL后训练中学习动态的机制。

2509.11963 2026-01-08 cs.CL 88%

ToolRM: Outcome Reward Models for Tool-Calling Large Language Models

ToolRM: 用于工具调用大型语言模型的成果奖励模型

Mayank Agarwal, Ibrahim Abdelaziz, Kinjal Basu, Merve Unuvar, Luis A. Lastras, Yara Rizk, Pavan Kapanipathi

机构 * IBM Research（IBM研究院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AI总结 ToolRM通过领域特定建模提升工具调用奖励模型性能，实现25%的改进并增强鲁棒性。

2505.11711 2025-12-19 cs.LG 88%

Reinforcement Learning Finetunes Small Subnetworks in Large Language Models

强化学习在大型语言模型中微调小型子网络

Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tur, Hao Peng

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

AI总结强化学习在大型语言模型中通过微调小型子网络显著提升性能，且该子网络更新稀疏性源于数据分布和正则化技术的影响。

Comments NeuRIPS 2025

2512.03882 2025-12-04 cs.LG 88%

Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Models

通过大语言模型实现少样本类增量学习的自动攻击发现

Haidong Kang, Wei Wu, Hanling Wang

机构 * School of Computer and Communication Engineering（计算机与通信工程学院）； School of Information and Communication（信息与通信学院）； University of Electronic Science and Technology of China（电子科学与技术大学）； Department of Strategic and Advanced Interdisciplinary Research（战略与先进跨学科研究部）； Pengcheng Laboratory（鹏城实验室）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

AI总结本文提出ACraft方法，利用大语言模型自动发现针对少样本类增量学习的最优攻击方法，有效提升攻击性能并降低成本。

2512.02807 2025-12-03 cs.CL 88%

SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment

SR-GRPO：稳定秩作为大语言模型对齐的内在几何奖励

Yixuan Tang, Yi Yang

机构 * The Hong Kong University of Science and Technology（香港理工大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

AI总结 SR-GRPO通过稳定秩作为内在几何奖励信号，无需外部监督提升大语言模型对齐性能。

2510.19474 2025-11-27 cs.LG 88%

g-DPO: Scalable Preference Optimization for Protein Language Models

g-DPO：可扩展的偏好优化用于蛋白质语言模型

Constance Ferragu, Jonathan D. Ziegler, Nicolas Deutschmann, Arthur Lindoulsi, Eli Bixby, Cradle ML Team

机构 * Cradle ML Team（Cradle团队）； Cradle Zürich, Switzerland（Cradle瑞士苏黎世）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

AI总结 g-DPO通过序列聚类和组近似方法提升蛋白质语言模型的偏好优化效率，实现更快的收敛速度和与标准DPO相当的性能。

Comments Accepted at two workshops: FM4LS NeurIPS 2025 (accepted-paper.html" target="_blank" rel="noopener">https://nips2025fm4ls.github.io/pages/accepted-paper.html) and MLSB in Copenhagen EurIPS 2025

2511.01450 2025-11-11 cs.CV cs.AI 88%

Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generation

Jie Du, Xinyu Gong, Qingshan Tan, Wen Li, Yangming Cheng, Weitao Wang, Chenlu Zhan, Suhui Wu, Hao Zhang, Jun Zhang

专题命中后训练与偏好优化：SFT(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments The paper is withdrawn due to the need for further revision and verification of experimental results. A revised version will be resubmitted once the updates are completed

2503.16365 2025-09-15 cs.CV cs.AI 88%

JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse

Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, Yitao Liang

机构 * Peking University（北京大学）； BIGAI（大疆创新）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；分类 cs.AI

Comments Accepted by ACL 2025

2509.00373 2025-09-03 cs.CV cs.AI 88%

Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models

Sihao Wu, Gaojie Jin, Wei Huang, Jianhong Wang, Xiaowei Huang

机构 * University of Liverpool（利物浦大学）； University of Exeter（埃克塞特大学）； Purple Mountain Laboratories（紫金山实验室）； University of Bristol（布里斯托大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

2504.05214 2025-08-26 cs.CL 88%

Post-Training Language Models for Continual Relation Extraction

Sefika Efeoglu, Adrian Paschke, Sonja Schimmler

机构 * Freie Universität Berlin（柏林自由大学）； Technische Universität Berlin（柏林技术大学）； Fraunhofer Institute for Open Communication Systems（弗劳恩霍夫开放通信系统研究所）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title)；large language model(abstract)；分类 cs.CL

Comments 17 pages, Initial Results and Reporting of the work. This work has been submitted to the IEEE for possible publication

2506.04746 2025-06-06 cs.LG 88%

Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models

Fei Ding, Baiqiao Wang, Zijian Zeng, Youwei Wang

机构 * Shenzhen Jinzheng Youzhi Technology（深圳金政智科技）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

2506.04463 2025-06-06 cs.CL 88%

Aligning Large Language Models with Implicit Preferences from User-Generated Content

Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang

机构 * University of Notre Dame（诺特大学）； Amazon.com Inc（亚马逊公司）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2025 Main Conference

2505.21061 2025-05-28 cs.CV cs.AI 88%

LPOI: Listwise Preference Optimization for Vision Language Models

Fatemeh Pesaran Zadeh, Yoojin Oh, Gunhee Kim

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(title)；RLHF(abstract)；分类 cs.AI

Comments ACL 2025 Main. Code is released at https://github.com/fatemehpesaran310/lpoi

2505.04993 2025-05-09 cs.CL 88%

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes

Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao

机构 * Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale Department of Computer Science Cranberry-Lemon University Pittsburgh, PA 15213, USA（计算机科学系， Cranberry-Lemon 大学）； Ji Q. Ren & Yevgeny LeNet Department of Computational Neuroscience University of the Witwatersrand Joburg, South Africa（计算神经科学系，威特沃特斯兰大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2410.10370 2025-04-14 cs.AI 88%

Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps

Han Wang, Yilin Zhao, Dian Li, Xiaohan Wang, Gang Liu, Xuguang Lan, Hui Wang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

2503.13262 2025-04-01 cs.CL 88%

TablePilot: Recommending Human-Preferred Tabular Data Analysis with Large Language Models

Deyin Yi, Yihao Liu, Lang Cao, Mengyu Zhou, Haoyu Dong, Shi Han, Dongmei Zhang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2404.09134 2024-12-11 cs.NI cs.LG 88%

Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission

Ruichen Zhang, Hongyang Du, Yinqiu Liu, Dusit Niyato, Jiawen Kang, Zehui Xiong, Abbas Jamalipour, Dong In Kim

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments 15 pages, 10 figures

2410.14155 2024-11-04 cs.CL 88%

Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models

Wei Jie Yeo, Ranjan Satapathy, Erik Cambria

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Under review

2410.17637 2024-10-24 cs.CV cs.AI 88%

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments Project URL: https://github.com/Liuziyu77/MIA-DPO

2410.16843 2024-10-23 cs.CL 88%

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

Zongmeng Zhang, Yufeng Shi, Jinhua Zhu, Wengang Zhou, Xiang Qi, Peng Zhang, Houqiang Li

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ICML 2024

Journal ref Proceedings of the 41st International Conference on Machine Learning, PMLR 235:59827-59850, 2024

2409.17534 2024-10-15 cs.AI 88%

Just Say What You Want: Only-prompting Self-rewarding Online Preference Optimization

Ruijie Xu, Zhihan Liu, Yongfei Liu, Shipeng Yan, Zhaoran Wang, Zhi Zhang, Xuming He

专题命中后训练与偏好优化：prompting(title,abstract)；preference optimization(title)；RLHF(abstract)；分类 cs.AI

2408.16090 2024-10-07 cs.LG 88%

EPO: Hierarchical LLM Agents with Environment Preference Optimization

Qi Zhao, Haotian Fu, Chen Sun, George Konidaris

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

Comments EMNLP 2024

2406.12606 2024-10-04 cs.CL 88%

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Jingyuan Wang, Ji-Rong Wen

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 14 pages, working in progress

2405.20215 2024-10-01 cs.CL 88%

TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models

Chen Zhang, Chengguang Tang, Dading Chong, Ke Shi, Guohua Tang, Feng Jiang, Haizhou Li

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP-2024 Findings

2409.07772 2024-09-13 cs.LG 88%

Alignment with Preference Optimization Is All You Need for LLM Safety

Reda Alami, Ali Khalifa Almansoori, Ahmed Alzubaidi, Mohamed El Amine Seddik, Mugariya Farooq, Hakim Hacid

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

2312.00738 2024-07-02 cs.CL 88%

SeaLLMs -- Large Language Models for Southeast Asia

Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, Lidong Bing

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Technical report, ACL 2024 DEMO TRACK

2401.06081 2024-06-18 cs.CL 88%

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 18 pages, Findings of ACL2024

2403.04187 2024-03-08 physics.bio-ph cs.AI cs.CE q-bio.BM 88%

Preference optimization of protein language models as a multi-objective binder design paradigm

Pouria Mistani, Venkatesh Mysore

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments Published at the GEM workshop, ICLR 2024. Generative and Experimental Perspectives for Biomolecular Design (https://www.gembio.ai/)

2302.07459 2023-02-21 cs.CL 88%

The Capacity for Moral Self-Correction in Large Language Models

Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, Jared Kaplan

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.CL

2209.06422 2022-09-15 cs.CL 88%

Language Chameleon: Transformation analysis between languages using Cross-lingual Post-training based on Pre-trained language models

Suhyune Son, Chanjun Park, Jungseob Lee, Midan Shim, Chanhee Lee, Yoonna Jang, Jaehyung Seo, Heuiseok Lim

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL

1. 后训练与偏好优化 4541 篇

Learning Dynamics in RL Post-Training for Language Models

ToolRM: Outcome Reward Models for Tool-Calling Large Language Models

Reinforcement Learning Finetunes Small Subnetworks in Large Language Models

Automatic Attack Discovery for Few-Shot Class-Incremental Learning via Large Language Models

SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment

g-DPO: Scalable Preference Optimization for Protein Language Models

Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generation

JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse

Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models

Post-Training Language Models for Continual Relation Extraction

Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models

Aligning Large Language Models with Implicit Preferences from User-Generated Content

LPOI: Listwise Preference Optimization for Vision Language Models

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes

Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps

TablePilot: Recommending Human-Preferred Tabular Data Analysis with Large Language Models

Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission

Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

Just Say What You Want: Only-prompting Self-rewarding Online Preference Optimization

EPO: Hierarchical LLM Agents with Environment Preference Optimization

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models

Alignment with Preference Optimization Is All You Need for LLM Safety

SeaLLMs -- Large Language Models for Southeast Asia

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint

Preference optimization of protein language models as a multi-objective binder design paradigm

The Capacity for Moral Self-Correction in Large Language Models

Language Chameleon: Transformation analysis between languages using Cross-lingual Post-training based on Pre-trained language models