大模型对齐与安全 - arXivDaily 专题

2602.10425 2026-02-12 cs.CV 78%

HII-DPO: Eliminate Hallucination via Accurate Hallucination-Inducing Counterfactual Images

HII-DPO: 通过准确的 hallucination-Inducing Counterfactual Images 消除幻觉

Yilin Yang, Zhenghui Guo, Yuke Wang, Omprakash Gnawali, Sheng Di, Chengming Zhang

机构 * University of Houston（休斯顿大学）； Rice University（里士满大学）； Argonne National Laboratory（阿贡国家实验室）

专题命中偏好对齐：DPO(title)；alignment(abstract)

AI总结 HII-DPO通过生成准确的幻觉诱导图像，有效缓解了VLMs在语言偏见下的幻觉问题，提升了模型的对齐能力。

2601.18457 2026-01-27 cs.IR 78%

Token-level Collaborative Alignment for LLM-based Generative Recommendation

基于令牌级的协同对齐用于基于大语言模型的生成推荐

Fake Lin, Binbin Hu, Zhi Zheng, Xi Zhu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Tong Xu

专题命中偏好对齐：alignment(title,abstract)

AI总结 TCA4Rec通过令牌级协同对齐框架，将协同过滤与大语言模型生成结合，提升推荐系统的准确性和可控性。

Comments 11 pages, 2 figures, 7 tables, WWW 2026

2512.22681 2026-01-01 cs.CV 78%

CritiFusion: Semantic Critique and Spectral Alignment for Faithful Text-to-Image Generation

CritiFusion: 语义批评与频谱对齐用于忠实的文本到图像生成

ZhenQi Chen, TsaiChing Ni, YuanFu Yang

机构 * National Yang Ming Chiao Tung University

专题命中偏好对齐：alignment(title,abstract)

AI总结 CritiFusion通过语义批评和频谱对齐提升文本到图像生成的忠实度和细节质量。

2510.10556 2025-10-20 cs.IR 78%

Self-Supervised Representation Learning with ID-Content Modality Alignment for Sequential Recommendation

Donglin Zhou, Weike Pan, Zhong Ming

专题命中偏好对齐：alignment(title,abstract)

Comments The article has been accepted by Frontiers of Computer Science (FCS), with the DOI: {10.1007/s11704-025-50269-4}

2504.14096 2025-09-26 cs.CV 78%

VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment

Yogesh Kulkarni, Pooyan Fazli

机构 * Arizona State University（亚利桑那州立大学）

专题命中偏好对齐：alignment(title,abstract)

Comments EMNLP 2025 (Main)

2405.13541 2025-09-18 cs.CL cs.AI cs.LG 78%

Annotation-Efficient Language Model Alignment via Diverse and Representative Response Texts

Yuu Jinnai, Ukyo Honda

机构 * CyberAgent / Tokyo, Japan（CyberAgent）

专题命中偏好对齐：alignment(title)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP Findings, 2025

2503.20309 2025-09-08 cs.CV 78%

Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs

Zitian Wang, Yue Liao, Kang Rong, Fengyun Rao, Yibo Yang, Si Liu

机构 * Beihang University（北航大学）； National University of Singapore（国立新加坡大学）； King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学）

专题命中偏好对齐：alignment(title,abstract)

Comments Accepted by ICCV 2025

2508.15388 2025-08-22 cs.IR 78%

TrackRec: Iterative Alternating Feedback with Chain-of-Thought via Preference Alignment for Recommendation

Yu Xia, Rui Zhong, Zeyu Song, Wei Yang, Junchen Wan, Qingpeng Cai, Chi Lu, Peng Jiang

专题命中偏好对齐：alignment(title,abstract)

2508.10116 2025-08-15 cs.IR 78%

Bridging Modality Gaps in e-Commerce Products via Vision-Language Alignment

Yipeng Zhang, Hongju Yu, Aritra Mandal, Canran Xu, Qunzhi Zhou, Zhe Wu

专题命中偏好对齐：alignment(title,abstract)

2502.09604 2025-06-17 cs.CL cs.AI cs.LG 78%

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

Yung-Sung Chuang, Benjamin Cohen-Wang, Shannon Zejiang Shen, Zhaofeng Wu, Hu Xu, Xi Victoria Lin, James Glass, Shang-Wen Li, Wen-tau Yih

机构 * Massachusetts Institute of Technology（麻省理工学院）； Meta FAIR

专题命中偏好对齐：alignment(title)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025 main conference paper. The source code is available at https://github.com/facebookresearch/SelfCite

2505.20627 2025-05-28 cs.GT stat.ML 78%

Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching

Zhekun Shi, Kaizhao Liu, Qi Long, Weijie J. Su, Jiancong Xiao

专题命中偏好对齐：alignment(title,abstract)

2503.18455 2025-03-25 cs.SE 78%

SEAlign: Alignment Training for Software Engineering Agent

Kechi Zhang, Huangzhao Zhang, Ge Li, Jinliang You, Jia Li, Yunfei Zhao, Zhi Jin

专题命中偏好对齐：alignment(title,abstract)

2502.06769 2025-03-19 cs.SE 78%

Enhancing Trust in Language Model-Based Code Optimization through RLHF: A Research Design

Jingzhi Gong

专题命中偏好对齐：RLHF(title)；trustworthy(abstract)

Comments Accepted by the Doctoral and Early Career Symposium (DECS) at ICSE 2025

2503.00591 2025-03-04 cs.CV 78%

AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models

Sohan Patnaik, Rishabh Jain, Balaji Krishnamurthy, Mausoom Sarkar

专题命中偏好对齐：alignment(title,abstract)

Comments Accepted for publication in CVPR 2025

2502.01943 2025-02-12 cs.CV 78%

DAMA: Data- and Model-aware Alignment of Multi-modal LLMs

Jinda Lu, Junkang Wu, Jinghan Li, Xiaojun Jia, Shuo Wang, YiFan Zhang, Junfeng Fang, Xiang Wang, Xiangnan He

专题命中偏好对齐：alignment(title)；DPO(abstract)

2406.11280 2025-01-09 cs.CV 78%

ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPO

Daechul Ahn, Yura Choi, San Kim, Youngjae Yu, Dongyeop Kang, Jonghyun Choi

专题命中偏好对齐：DPO(title,abstract)

Comments AAAI 2025

2409.10157 2024-09-17 eess.AS cs.SD eess.SP 78%

Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization

Xiaoxue Gao, Chen Zhang, Yiming Chen, Huayun Zhang, Nancy F. Chen

专题命中偏好对齐：DPO(title,abstract)

Comments 5 pages

2408.10433 2024-08-21 cs.CV 78%

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

Yassine Ouali, Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos

专题命中偏好对齐：DPO(title,abstract)

Comments Accepted at ECCV 2024

2405.03803 2024-05-08 cs.CV 78%

MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization

Massimiliano Pappa, Luca Collorone, Giovanni Ficarra, Indro Spinelli, Fabio Galasso

专题命中偏好对齐：alignment(title)；DPO(abstract)

2305.11206 2023-05-22 cs.CL cs.AI cs.LG 78%

LIMA: Less Is More for Alignment

Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy

专题命中偏好对齐：alignment(title)；分类 cs.CL、cs.AI、cs.LG

2604.06621 2026-08-26 cs.GL cs.LG stat.ML 版本更新 77%

The Theorems of Dr. David Blackwell and Their Contributions to Artificial Intelligence

大卫·布莱克韦尔定理及其对人工智能的贡献

Napoleon Paxton

机构 * School of Information, University of California, Berkeley（加州大学伯克利分校信息学院）

专题命中偏好对齐：RLHF(abstract,abstract_cn)；alignment(abstract)；分类 cs.LG

AI总结本文综述了布莱克韦尔的三个重要定理及其对现代人工智能和机器学习的影响，包括马尔可夫链蒙特卡罗推断、自主机器人导航、生成模型训练等领域的应用。

Comments Survey article, 20 pages, 1 figure, 2 tables

2608.21993 2026-08-25 cs.LG 新提交 77%

Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

门控解耦组合多臂老虎机：带监督校准动作缩放与预执行门控的上下文多臂老虎机统一理论

Oleg Miroshnichenko

专题命中偏好对齐：RLHF(abstract,abstract_cn)；safety(abstract)；分类 cs.LG

AI总结该研究提出门控解耦组合多臂老虎机（GDCB），证明其可统一多个工业系统，核心定理可将非平稳问题转化为近似平稳问题，还推广了相关结果，配套论文已验证短期租赁动态定价实例。

Comments 30 pages, 4 figures

2608.06632 2026-08-10 cs.AI 新提交 77%

Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation

塑造你的信息流：一种基于大语言模型的智能体对话推荐系统

Ziyun Xu, Bosen Ding, Yue Zhang, Ji Qi, Qingyuan Song, Jizhou Huang, Liwei Wang, Jefferey Santelli, Yue Weng, Qichao Que, Zhenheng Yang, Junfeng Pan, Linhong Zhu

机构 * Meta Platforms（元平台公司）

专题命中偏好对齐：DPO(abstract,abstract_cn)；alignment(abstract)；分类 cs.AI

AI总结本文提出基于LLM的智能体推荐框架SYF，采用三层架构实现实时多模态内容协同策划，经离线评估与在线A/B实验验证，可提升信息流相关性与用户情感，为工业场景提供交互式推荐方案。

Comments Accepted in RecSys 2026 Industrial Track

2606.27771 2026-08-07 cs.LG cs.CV 版本更新 77%

NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

NormGuard: 流匹配强化学习中保持奖励的范数约束

Tianlin Pan, Lianyu Pang, Cheng Da, Huan Yang, Changqian Yu, Kun Gai, Wenhan Luo

机构 * The Hong Kong University of Science and Technology（香港科技大学）； Kuaishou Technology（快手科技）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中偏好对齐：DPO(abstract,abstract_cn)；alignment(abstract)；分类 cs.LG

AI总结针对流生成器强化学习后训练中感知质量下降的问题，提出NormGuard方法，通过铰链惩罚约束速度范数，在保持奖励的同时提升图像质量和真实性。

Comments v5: Fixed PDF rendering compatibility issue affecting Apple PDFKit (macOS Preview/iOS PDF viewer). No changes to technical content compared to v4

2504.12501 2026-08-04 cs.LG 77%

Reinforcement Learning from Human Feedback

基于人类反馈的强化学习

Nathan Lambert

机构 * Nathan Lambert（纳瑟恩·拉姆伯特）

专题命中偏好对齐：RLHF(abstract,abstract_cn)；alignment(abstract)；分类 cs.LG

AI总结本书系统介绍强化学习从人类反馈的核心方法，涵盖指令微调、奖励模型训练及拒绝采样等关键技术，探讨合成数据与评估中的前沿问题。

Comments 239 pages. Web-native version at https://rlhfbook.com/ Continually improving, latest version at website

2603.00025 2026-08-04 cs.CL 版本更新 77%

TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation

TAB-PO：面向Token关键结构化生成的具有Token级自适应障碍的偏好优化

Samah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Sreeraj Ramachandran, Elyas Irankhah, Aimee Kendall Roundtree

机构 * Yale University（耶鲁大学）； Texas State University（德克萨斯州立大学）

专题命中偏好对齐：DPO(abstract,abstract_cn)；alignment(abstract)；分类 cs.CL

AI总结针对结构化预测中偏好与拒绝对象仅少数token不同导致的梯度稀释和token侵蚀问题，提出基于混淆感知偏好构建和Token级自适应障碍的TAB-PO方法，在SciERC任务上显著提升关键指标。

2607.29185 2026-08-03 cs.CL 新提交 77%

Learning Latent Reasoning Traces for Scalar Reward Models End-to-End

学习标量奖励模型的端到端潜在推理轨迹

Sanwoo Lee, Clive Bai, Hsiu-Yuan Huang, Kun Liang, Weijie Liu, Yunfang Wu

专题命中偏好对齐：RLHF(abstract,abstract_cn)；alignment(abstract)；分类 cs.CL

AI总结针对奖励模型范式不匹配问题，提出LatentRM框架，将推理轨迹作为潜在变量端到端优化，在多任务上优于各类基准奖励模型。

2607.28647 2026-08-03 cs.HC cs.AI 新提交 77%

ConnectED: A Curriculum-Aligned AI System for Vietnamese Instructional Lesson Planning and Student Learning

ConnectED：面向越南教学课程规划与学生学习的课程对齐AI系统

Thang Doan Viet, Anh Nguyen Hoang, Tinh Luong Son, Anh Hoang Thi Ngoc, Huyen Giang Thi Thu, Tai Le Quy

专题命中偏好对齐：DPO(abstract,abstract_cn)；alignment(abstract)；分类 cs.AI

AI总结该研究提出以人为本的AI系统ConnectED，基于VietEduQwen构建，支持越南教育完整教学生命周期，经评估其准确率优于基准模型，可缩短教师备课时间且获师生高满意度。

2607.26891 2026-07-30 cs.CL 新提交 77%

DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

DIRECT：基于大语言模型的高效对齐序列标注直接解码方法

Yilei Wang, Jiaxin Gan, Kexuan Zhang, Ling Li, Wentao Zhang, Peichao Lai

专题命中偏好对齐：DPO(abstract,abstract_cn)；alignment(abstract)；分类 cs.CL

AI总结针对现有大语言模型序列标注方法的领域对齐不足与推理效率低问题，提出DIRECT框架，结合训练时优化与推理时校正，在8个数据集上实现性能与效率的显著提升。

2602.00846 2026-07-08 cs.CL 版本更新 77%

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis

Omni-RRM：通过基于自动评分标准的偏好合成推进全模态奖励建模

Zicheng Kong, Dehua Ma, Zhenbo Xu, Alven Yang, Yiwei Ru, Haoran Wang, Zixuan Zhou, Fuqing Bie, Liuyu Xiang, Huijia Wu, Jian Zhao, Zhaofeng He

机构 * Beijing University of Posts and Telecommunications（北京邮电大学）； Tsinghua University（清华大学）； Institute of Artificial Intelligence (TeleAI)（人工智能研究所）； ShiFang Technology Inc.（ShiFang科技公司）

专题命中偏好对齐：RLHF(abstract,abstract_cn)；alignment(abstract)；分类 cs.CL

AI总结研究针对多模态大语言模型对齐难题，提出全模态基于评分标准的奖励模型Omni-RRM。通过自动评分标准偏好合成构建数据集，用特定训练方案提升奖励辨别力，在多模态基准测试中达最优精度，还能指导选择与迁移，提升模型对齐能力。

Comments ECCV 2026

1. 偏好对齐 3266 篇

HII-DPO: Eliminate Hallucination via Accurate Hallucination-Inducing Counterfactual Images

Token-level Collaborative Alignment for LLM-based Generative Recommendation

CritiFusion: Semantic Critique and Spectral Alignment for Faithful Text-to-Image Generation

Self-Supervised Representation Learning with ID-Content Modality Alignment for Sequential Recommendation

VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment

Annotation-Efficient Language Model Alignment via Diverse and Representative Response Texts

Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs

TrackRec: Iterative Alternating Feedback with Chain-of-Thought via Preference Alignment for Recommendation

Bridging Modality Gaps in e-Commerce Products via Vision-Language Alignment

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching

SEAlign: Alignment Training for Software Engineering Agent

Enhancing Trust in Language Model-Based Code Optimization through RLHF: A Research Design

AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models

DAMA: Data- and Model-aware Alignment of Multi-modal LLMs

ISR-DPO: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective DPO

Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization

LIMA: Less Is More for Alignment

The Theorems of Dr. David Blackwell and Their Contributions to Artificial Intelligence

Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

Shape Your Feed: An LLM-based Agentic System for Conversational Recommendation

NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

Reinforcement Learning from Human Feedback

TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation

Learning Latent Reasoning Traces for Scalar Reward Models End-to-End

ConnectED: A Curriculum-Aligned AI System for Vietnamese Instructional Lesson Planning and Student Learning

DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis