语言大模型 / LLM - arXivDaily 专题

2602.05205 2026-02-06 cs.CL cs.AI 90%

Aligning Large Language Model Behavior with Human Citation Preferences

使大型语言模型行为与人类引用偏好对齐

Kenichiro Ando, Tatsuya Harada

机构 * RIKEN AIP（日本研究所）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(title)；LLM(abstract)；preference optimization(abstract)

AI总结本研究通过构建数据集分析LLM引用行为与人类偏好的一致性，发现模型在引用医学文本和数字句子上存在偏差，通过优化可提升对齐效果。

Comments Work In Progress

2510.04217 2026-02-03 cs.LG cs.AI 90%

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

MLLMEraser: 通过激活引导在多模态大语言模型中实现测试时遗忘

Chenlu Ding, Jiancan Wu, Leheng Sheng, Fan Zhang, Yancheng Yuan, Xiang Wang, Xiangnan He

机构 * University of Science and Technology of China（中国科学技术大学）； National University of Singapore（新加坡国立大学）； The Hong Kong Polytechnic University（香港理工大学）； Huawei（华为）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

AI总结 MLLMEraser通过激活引导实现多模态大语言模型的测试时遗忘，无需训练且有效降低计算成本，同时保持保留知识的效用。

2601.17096 2026-01-27 cs.CY cs.AI cs.CL 90%

Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models

超越工具性和替代性范式：引入机器文化作为大型语言模型中的涌现现象

Yueqing Hu, Xinyang Peng, Yukun Zhao, Lin Qiu, Ka-lai Hung, Kaiping Peng

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

AI总结本研究提出机器文化作为大型语言模型中的一种新兴现象，挑战传统工具性和替代性范式，揭示模型在文化表现上的独特特性。

Comments 16 pages, 6 figures

2601.01828 2026-01-06 cs.CL cs.AI 90%

Emergent Introspective Awareness in Large Language Models

大语言模型中的涌现自我意识

Jack Lindsey

机构 * Anthropic

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI

AI总结研究发现大语言模型在特定条件下能识别自身内部状态，但其自我意识仍不成熟且依赖上下文。

2512.06097 2025-12-09 cs.CL cs.AI 90%

Empathy by Design: Aligning Large Language Models for Healthcare Dialogue

设计中的共情：为医疗对话对齐大语言模型

Emre Umucu, Guillermina Solis, Leon Garza, Emilia Rivas, Beatrice Lee, Anantaa Kotal, Aritran Piplai

机构 * Department of Public Health Sciences, The University of Texas at El Paso, USA（公共卫生科学系，德克萨斯理工大学埃尔帕索分校）； Department of Nursing, The University of Texas at El Paso, USA（护理系，德克萨斯理工大学埃尔帕索分校）； Department of Rehabilitation Sciences, The University of Texas at El Paso, USA（康复科学系，德克萨斯理工大学埃尔帕索分校）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

AI总结本文提出基于DPO的对齐框架，通过优化医疗对话中事实准确性、语义连贯性和共情能力，提升AI助手在医疗场景中的可信度和人文关怀。

2510.23658 2025-11-13 cs.LG cs.AI 90%

Aligning Diffusion Language Models via Unpaired Preference Optimization

Vaibhav Jindal, Hejian Sang, Chun-Mao Lai, Yanning Chen, Zhipeng Wang

机构 * LinkedIn Corporation（LinkedIn公司）； University of California San Diego（加州大学圣地亚哥分校）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

2410.12854 2025-10-27 cs.CL cs.AI 90%

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

Weibin Liao, Xu Chu, Yasha Wang

机构 * School of Computer Science, Peking University（北京大学计算机科学系）； Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）； National Research and Engineering Center of Software Engineering, Peking University（北京大学软件工程研究中心）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICLR 2025

2506.23601 2025-09-30 cs.CL cs.AI 90%

Semantic-guided Diverse Decoding for Large Language Model

Weijie Shi, Yue Cui, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Mengze Li, Sirui Han, Jia Zhu, Jiajie Xu, Xiaofang Zhou

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

2506.04039 2025-09-23 cs.CV cs.AI cs.CL 90%

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Jiulong Wu, Zhengliang Shi, Shuaiqiang Wang, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao, Min Zhang

机构 * Soochow University（苏州大学）； Baidu Inc.（百度公司）； Shandong University（山东大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments This paper is accepted by EMNLP2025

2509.08329 2025-09-11 cs.LG cs.AI 90%

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

Lukas Toral, Teddy Lazebnik

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

2509.06980 2025-09-10 cs.LG cs.AI 90%

RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use

Jiajun Chai, Guojun Yin, Zekun Xu, Chuhuai Yue, Yi Jia, Siyu Xia, Xiaohan Wang, Jiwen Jiang, Xiaoguang Li, Chengqi Dong, Hang He, Wei Lin

机构 * RLFactory Team（RLFactory团队）

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

2508.15790 2025-08-25 cs.CL cs.AI 90%

KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration

Nan Wang, Yongqi Fan, yansha zhu, ZongYu Wang, Xuezhi Cao, Xinyan He, Haiyun Jiang, Tong Ruan, Jingping Liu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2505.19743 2025-08-19 cs.CL cs.LG 90%

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models

Yang Zhang, Yu Yu, Bo Tang, Yu Zhu, Chuxiong Sun, Wenqiang Wei, Jie Hu, Zipeng Xie, Zhiyu Li, Feiyu Xiong, Edward Chung

机构 * Hong Kong Polytechnic University（香港理工大学）； MemTensor (Shanghai) Technology Co., Ltd（MemTensor（上海）科技有限公司）； University of Science and Technology of China（中国科学技术大学）； China Telecom Corporation Limited Beijing Research Institute（中国电信北京研究院）； Nanjing University of Information Science and Technology（南京信息工程大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted to 34th International Joint Conference on Artificial Intelligence (IJCAI 2025)

2508.01174 2025-08-05 cs.LG cs.AI 90%

RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models

Kaichen Zhang, Shenghao Gao, Yuzhong Hong, Haipeng Sun, Junwei Bao, Hongfei Jiang, Yang Song, Hong Dingqian, Hui Xiong

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））； Zuoyebang Education Technology（左就业教育科技）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；分类 cs.AI、cs.LG

2507.17107 2025-07-30 cs.LG cs.AI 90%

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Andrii Balashov

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments The manuscript has been withdrawn due to significant overlap in methodology and results with a prior work (arXiv:2505.11711) that we were not aware of at the time of submission. To maintain academic integrity and avoid redundancy in the literature, we have chosen to withdraw this version

2507.04099 2025-07-16 cs.CL cs.AI 90%

Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching

Thomas Savage

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2507.07725 2025-07-11 cs.CL cs.AI 90%

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization

Zhijin Dong

机构 * Peking University（北京大学）

专题命中后训练与偏好优化：post-training(title,abstract)；preference optimization(title)；large language model(abstract)；language model(abstract)

2503.13377 2025-07-01 cs.CV cs.AI cs.CL 90%

Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding

Ye Wang, Ziheng Wang, Boshen Xu, Yang Du, Kejun Lin, Zihan Xiao, Zihao Yue, Jianzhong Ju, Liang Zhang, Dingyi Yang, Xiangnan Fang, Zewen He, Zhenbo Luo, Wenxuan Wang, Junqi Lin, Jian Luan, Qin Jin

机构 * AIM3 Lab, Renmin University of China（中国人民大学AIM3实验室）； MiLM Plus, Xiaomi Inc.（小米公司MiLM Plus）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Comments Project Page: https://xuboshen.github.io/Time-R1/

2506.03541 2025-06-05 cs.CL cs.AI 90%

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement

Xiaofeng Zhou, Heyan Huang, Lizi Liao

机构 * Beijing Institute of Technology（北京理工大学）； Singapore Management University（新加坡国立大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 10 figures. The camera-ready paper for Findings of ACL 2025

2501.04945 2025-06-03 cs.CL cs.AI 90%

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models

Qingyu Ren, Jie Zeng, Qianyu He, Jiaqing Liang, Yanghua Xiao, Weikang Zhou, Zeye Sun, Fei Yu

机构 * Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（上海数据科学 key laboratory，计算机科学与人工智能学院，复旦大学）； School of Data Science, Fudan University（数据科学学院，复旦大学）； Ant Group（蚂蚁集团）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2502.11051 2025-05-28 cs.CL cs.AI 90%

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

Jiahao Huo, Yibo Yan, Xu Zheng, Yuanhuiyi Lyu, Xin Zou, Zhihua Wei, Xuming Hu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted as ACL 2025 Findings

2505.12435 2025-05-20 cs.LG cs.AI cs.DC 90%

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment

Wenqiao Zhu, Ji Liu, Lulu Wang, Jun Wu, Yulun Zhang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, to appear in ACL'25

2410.12851 2025-04-22 cs.CL cs.AI 90%

VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models

Lisa Dunlap, Krishna Mandal, Trevor Darrell, Jacob Steinhardt, Joseph E Gonzalez

机构 * UC Berkeley（加州大学伯克利分校）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments unironic use of the word 'vibe', added more analysis and cooler graphs. added website link

2410.01532 2025-04-01 cs.CL cs.AI cs.CV cs.HC 90%

Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models

Angela Lopez-Cardona, Carlos Segura, Alexandros Karatzoglou, Sergi Abadal, Ioannis Arapakis

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted to ICLR 2025

2503.03039 2025-03-06 cs.LG cs.AI 90%

LLM Misalignment via Adversarial RLHF Platforms

Erfan Entezami, Ali Naseh

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

2410.02205 2025-02-11 cs.CL cs.AI cs.LO 90%

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

Yinhong Liu, Zhijiang Guo, Tianya Liang, Ehsan Shareghi, Ivan Vulić, Nigel Collier

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2402.13846 2025-02-04 cs.AI cs.CL cs.CR 90%

Large Language Models are Advanced Anonymizers

Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments International Conference on Learning Representations (ICLR 2024)

2412.14516 2024-12-20 cs.LG cs.CL 90%

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment

Teng Xiao, Yige Yuan, Huaisheng Zhu, Mingxiao Li, Vasant G Honavar

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted by NeurIPS 2024 Main

2412.13702 2024-12-20 cs.CL cs.AI 90%

Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

Kunat Pipatanakul, Potsawee Manakul, Natapong Nitarach, Warit Sirichotedumrong, Surapon Nonesung, Teetouch Jaknamon, Parinthapat Pengpun, Pittawat Taveekitworachai, Adisai Na-Thalang, Sittipong Sripaisarnmongkol, Krisanapong Jirayoot, Kasima Tharnpipitchai

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI

Comments technical report, 55 pages

2406.10023 2024-10-30 cs.LG cs.CL stat.ML 90%

Deep Bayesian Active Learning for Preference Modeling in Large Language Models

Luckeciano C. Melo, Panagiotis Tigas, Alessandro Abate, Yarin Gal

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

1. 后训练与偏好优化 4535 篇

Aligning Large Language Model Behavior with Human Citation Preferences

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models

Emergent Introspective Awareness in Large Language Models

Empathy by Design: Aligning Large Language Models for Healthcare Dialogue

Aligning Diffusion Language Models via Unpaired Preference Optimization

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

Semantic-guided Diverse Decoding for Large Language Model

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use

KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models

RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization

Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment

VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models

Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models

LLM Misalignment via Adversarial RLHF Platforms

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

Large Language Models are Advanced Anonymizers

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment

Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

Deep Bayesian Active Learning for Preference Modeling in Large Language Models