语言大模型 / LLM - arXivDaily 专题

2407.02302 2025-02-19 cs.CL 89%

Towards Human Understanding of Paraphrase Types in Large Language Models

Dominik Meier, Jan Philip Wahle, Terry Ruas, Bela Gipp

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；RLHF(abstract)；prompting(abstract)

Journal ref Proceedings of the 31st International Conference on Computational Linguistics (2025), pages 6298-6316

2502.06823 2025-02-13 cs.LG cs.CV cs.GR cs.IR 89%

CTR-Driven Advertising Image Generation with Multimodal Large Language Models

Xingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang, Yanyin Chen, Haohan Wang, Linkai Liu, Yaoyu Li, Jinyuan Zhao, Yu Li, Zheng Zhang, Jingjing Lv, Junjie Shen, Zhangang Lin, Jingping Shao, Yuanjie Shao, Xinge You, Changxin Gao, Nong Sang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.LG

Comments Accepted to WWW 2025

2502.04306 2025-02-07 cs.CL 89%

ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization

Yinjie Wang, Ling Yang, Guohao Li, Mengdi Wang, Bryon Aragam

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

Comments Project: https://github.com/Gen-Verse/ScoreFlow

2412.15650 2024-12-23 cs.LG 89%

Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution

Wentao Tan, Qiong Cao, Yibing Zhan, Chao Xue, Changxing Ding

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.LG

Comments AAAI 2025. The code is available at https://github.com/WentaoTan/SENA

2409.12822 2024-12-10 cs.CL 89%

Language Models Learn to Mislead Humans via RLHF

Jiaxin Wen, Ruiqi Zhong, Akbir Khan, Ethan Perez, Jacob Steinhardt, Minlie Huang, Samuel R. Bowman, He He, Shi Feng

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(title,abstract)；post-training(abstract)；分类 cs.CL

2405.17374 2024-11-01 cs.LG 89%

Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models

ShengYun Peng, Pin-Yu Chen, Matthew Hull, Duen Horng Chau

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments NeurIPS'24

2404.02655 2024-10-10 cs.CL 89%

Calibrating the Confidence of Large Language Models by Eliciting Fidelity

Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng Yan, Yaqian Zhou, Xipeng Qiu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments EMNLP 2024

2406.04481 2024-10-10 cs.AI 89%

Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF

Yuan Sun, Navid Salami Pargoo, Peter J. Jin, Jorge Ortiz

专题命中后训练与偏好优化：RLHF(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

2312.07401 2024-10-08 cs.AI 89%

On Diversified Preferences of Large Language Model Alignment

Dun Zeng, Yong Dai, Pengyu Cheng, Longyue Wang, Tianhao Hu, Wanshun Chen, Nan Du, Zenglin Xu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments EMNLP 2024

2404.00978 2024-09-19 cs.CL 89%

Prior Constraints-based Reward Model Training for Aligning Large Language Models

Hang Zhou, Chenglong Wang, Yimin Hu, Tong Xiao, Chunliang Zhang, Jingbo Zhu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by CCL 2024

2407.16686 2024-08-26 cs.CL 89%

Can Large Language Models Automatically Jailbreak GPT-4V?

Yuanwei Wu, Yue Huang, Yixin Liu, Xiang Li, Pan Zhou, Lichao Sun

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments TrustNLP@NAACL2024 (Fourth Workshop on Trustworthy Natural Language Processing)

2402.11907 2024-08-16 cs.CL 89%

Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation

Aiwei Liu, Haoping Bai, Zhiyun Lu, Xiang Kong, Simon Wang, Jiulong Shan, Meng Cao, Lijie Wen

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments 24 pages, 5 pages

2310.07820 2024-08-13 cs.LG 89%

Large Language Models Are Zero-Shot Time Series Forecasters

Nate Gruver, Marc Finzi, Shikai Qiu, Andrew Gordon Wilson

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.LG

Comments NeurIPS 2023. Code available at: https://github.com/ngruver/llmtime

2406.15178 2024-06-24 cs.CL 89%

Hybrid Alignment Training for Large Language Models

Chenglong Wang, Hang Zhou, Kaiyan Chang, Bei Li, Yongyu Mu, Tong Xiao, Tongran Liu, Jingbo Zhu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments accepted by ACL (Findings) 2024

2311.04155 2024-06-24 cs.CL 89%

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training

Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, Minlie Huang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to ACL 2024

2312.12999 2024-06-04 cs.CL 89%

Machine Mindset: An MBTI Exploration of Large Language Models

Jiaxi Cui, Liuzhenghao Lv, Jing Wen, Rongsheng Wang, Jing Tang, YongHong Tian, Li Yuan

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

2404.12715 2024-05-31 cs.CL 89%

Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration

Yichong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Bing Qin, Ting Liu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 16 pages, 9 figures, 9 tables

2404.06390 2024-04-16 cs.CL 89%

Latent Distance Guided Alignment Training for Large Language Models

Haotian Luo

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL

2403.04642 2024-03-08 cs.LG 89%

Teaching Large Language Models to Reason with Reinforcement Learning

Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, Roberta Raileanu

专题命中后训练与偏好优化：large language model(title)；language model(title)；LLM(abstract)；SFT(abstract)

2309.05958 2024-02-08 cs.CL cs.CY cs.HC 89%

The Moral Machine Experiment on Large Language Models

Kazuhiro Takemoto

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 12 pages, 2 Figures

Journal ref Royal Society Open Science 11 (2), 231393 (2024)

2307.07164 2024-01-29 cs.CL cs.IR 89%

Learning to Retrieve In-Context Examples for Large Language Models

Liang Wang, Nan Yang, Furu Wei

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by EACL 2024

2310.11564 2023-10-19 cs.CL 89%

Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging

Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, Prithviraj Ammanabrolu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments Preprint

2310.07629 2023-10-12 cs.CL cs.CY 89%

The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values

Hannah Rose Kirk, Andrew M. Bean, Bertie Vidgen, Paul Röttger, Scott A. Hale

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted for the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP, Main)

2608.16068 2026-08-18 cs.CL cs.AI 新提交 89%

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

CAPO：面向大语言模型智能体的感知约束提示优化

Victor Ye Dong, Reid Pryzant, Yi Liu, Jian Jiao

机构 * Microsoft（微软）

专题命中后训练与偏好优化：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本研究提出CAPO与DCAPO两种方法，通过原始对偶框架优化LLM智能体的系统提示，在智能体及助手式任务中均提升了可行性与性能。

2509.23102 2026-08-12 cs.AI cs.CL 版本更新 89%

Multiplayer Nash Preference Optimization

多玩家纳什偏好优化

Fang Wu, Xu Huang, Weihao Xuan, Zhiwei Zhang, Yijia Xiao, Guancheng Wan, Xiaomin Li, Bing Hu, Peng Xia, Jure Leskovec, Yejin Choi

机构 * Stanford University（斯坦福大学）； Georgia Institute of Technology（佐治亚理工学院）； The University of Tokyo（东京大学）； RIKEN AIP（日本理化学研究所智能系统研究中心）； Pennsylvania State University（宾夕法尼亚州立大学）； University of California, Los Angeles（加州大学洛杉矶分校）； Harvard University（哈佛大学）； UNC–Chapel Hill（北卡罗来纳大学教堂山分校）

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract,abstract_cn)；large language model(abstract)；language model(abstract)

AI总结本文提出多玩家纳什偏好优化框架，扩展了传统的两玩家纳什对齐方法，通过引入多玩家博弈机制，提升对复杂非传递性人类偏好的对齐能力，并在多个基准测试中表现更优。

Journal ref ICLR 2026 Oral

2607.20448 2026-07-24 cs.CL cs.LG 新提交 89%

Domyn-Small: A European 10B Reasoning Language Model

Domyn-Small：一个欧洲的100亿参数推理语言模型

Simone Angarano, Francesco Bertolotti, Federico D'Ambrosio, Michele Resta, Alessandro Rognoni, Nicolò Ruggeri, Dario Salvati, Andrea Valenti, Alberto Veneri, Martin Cimmino

机构 * CINECA（意大利国家计算应用研究所）

专题命中后训练与偏好优化：language model(title,abstract)；SFT(abstract,abstract_cn)；LLM(abstract)；post-training(abstract)

AI总结介绍了基于9万亿令牌多语言数据预训练的100亿参数推理语言模型Domyn-Small，经持续预训练、监督微调、强化学习等训练后管道，实现双模式推理，在与对等模型对比中平衡了准确性与效率，还发布了相关权重及开源框架。

Comments 27 pages, 5 figure

2607.15095 2026-07-20 cs.CL cs.AI cs.MA 版本更新 89%

Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents

数字万神殿：使用大语言模型智能体模拟和审计联盟形成

Dylan Van Mulders, Matthias Bogaert, Dirk Van den Poel

机构 * Ghent University（根特大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；SFT(abstract)

AI总结该研究针对政治联盟形成谈判，提出结合多种技术的多智能体框架，在弗拉芒选举中实施，通过引入相关拓扑、分数和检验使其可解释，模拟产生稳定结果，能可靠预测现实，提供了探索政党兼容性等的测试平台。

Comments 11 pages, 2 figures, 5 tables, To be published in the AIDEM Workshop proceedings of the ECML PKDD 2026 Conference

2606.21090 2026-06-23 cs.AI cs.LG 新提交 89%

Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training

自我改进可能导致自我退化：LLM自我训练的兴起与崩溃失败模式

Jianzhe Lin

机构 * MetaAI

专题命中后训练与偏好优化：LLM(title,title_cn)；post-training(abstract)；分类 cs.AI、cs.LG

AI总结研究LLM自我训练中pass@1先升后降的崩溃模式，发现非跨任务遗忘而是策略过度优化，提出CARE、ES和GRPO三种控制方法，在Qwen-2.5模型上验证效果。

Comments 31 pages

2606.09961 2026-06-10 cs.LG cs.AI 新提交 89%

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

3SPO: 面向LLM智能体的状态分数监督策略优化

Yu Han, Kailing Li, Yang Jiao, Yulin Dai, Yuqian Fu, Linhai Zhuo, Tianwen Qian

机构 * School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）； Fudan University（复旦大学）； KAUST（卡塔尔人工智能研究学院）； Fuzhou University（福州大学）

专题命中后训练与偏好优化：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结提出3SPO算法，通过动态状态分数监督实现逐步骤策略优化，解决多轮智能体任务中奖励稀疏和信用分配问题，在ALFWorld和WebShop上分别比GRPO提升22.6%和15.6个百分点。

2506.03106 2026-06-09 cs.CL cs.AI 89%

Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback

Critique-GRPO：通过自然语言和数值反馈提升大语言模型推理能力

Xiaoying Zhang, Yipeng Zhang, Hao Sun, Kaituo Feng, Chaochao Lu, Chao Yang, Helen Meng

机构 * HCCL, The Chinese University of Hong Kong, Hong Kong, China（香港中文大学人工智能研究中心，香港，中国）； University of Cambridge, Cambridge, United Kingdom（剑桥大学，剑桥，英国）； MMLab, The Chinese University of Hong Kong, Hong Kong, China（香港中文大学人工智能实验室，香港，中国）； Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室，上海，中国）

专题命中后训练与偏好优化：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本文提出Critique-GRPO框架，结合自然语言和数值反馈提升LLM推理能力，实验显示其在多个任务中优于传统方法，显著提升推理性能。

Comments Accepted by ICML 2026 Spotlight

1. 后训练与偏好优化 4541 篇

Towards Human Understanding of Paraphrase Types in Large Language Models

CTR-Driven Advertising Image Generation with Multimodal Large Language Models

ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization

Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution

Language Models Learn to Mislead Humans via RLHF

Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models

Calibrating the Confidence of Large Language Models by Eliciting Fidelity

Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF

On Diversified Preferences of Large Language Model Alignment

Prior Constraints-based Reward Model Training for Aligning Large Language Models

Can Large Language Models Automatically Jailbreak GPT-4V?

Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation

Large Language Models Are Zero-Shot Time Series Forecasters

Hybrid Alignment Training for Large Language Models

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training

Machine Mindset: An MBTI Exploration of Large Language Models

Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration

Latent Distance Guided Alignment Training for Large Language Models

Teaching Large Language Models to Reason with Reinforcement Learning

The Moral Machine Experiment on Large Language Models

Learning to Retrieve In-Context Examples for Large Language Models

Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging

The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values

CAPO: Constraint-Aware Prompt Optimization for LLM Agents

Multiplayer Nash Preference Optimization

Domyn-Small: A European 10B Reasoning Language Model

Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents

Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback