大模型对齐与安全

A Versatile Multimodal Agent for Multimedia Content Generation

多功能多模态代理用于多媒体内容生成

Daoan Zhang, Wenlin Yao, Xiaoyang Wang, Yebowen Hu, Jiebo Luo, Dong Yu

机构 * University of Rochester（罗切斯特大学） ； Tencent AI Lab, Bellevue（腾讯AI实验室） ； University of Central Florida（中央佛罗里达大学） 纠错

专题命中 偏好对齐 ：alignment(abstract)

AI总结 本文提出了一种多功能多模态代理，通过技能获取理论和两阶段相关策略，实现复杂多媒体内容生成任务的自动化。

Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model

Seedance 1.5 pro: 一种原生音频视频联合生成基础模型

Team Seedance, Heyi Chen, Siyan Chen, Xin Chen, Yanfei Chen, Ying Chen, Zhuo Chen, Feng Cheng, Tianheng Cheng, Xinqi Cheng, Xuyan Chi, Jian Cong, Jing Cui, Qinpeng Cui, Qide Dong, Junliang Fan, Jing Fang, Zetao Fang, Chengjian Feng, Han Feng, Mingyuan Gao, Yu Gao, Dong Guo, Qiushan Guo, Boyang Hao, Qingkai Hao, Bibo He, Qian He, Tuyen Hoang, Ruoqing Hu, Xi Hu, Weilin Huang, Zhaoyang Huang, Zhongyi Huang, Donglei Ji, Siqi Jiang, Wei Jiang, Yunpu Jiang, Zhuo Jiang, Ashley Kim, Jianan Kong, Zhichao Lai, Shanshan Lao, Yichong Leng, Ai Li, Feiya Li, Gen Li, Huixia Li, JiaShi Li, Liang Li, Ming Li, Shanshan Li, Tao Li, Xian Li, Xiaojie Li, Xiaoyang Li, Xingxing Li, Yameng Li, Yifu Li, Yiying Li, Chao Liang, Han Liang, Jianzhong Liang, Ying Liang, Zhiqiang Liang, Wang Liao, Yalin Liao, Heng Lin, Kengyu Lin, Shanchuan Lin, Xi Lin, Zhijie Lin, Feng Ling, Fangfang Liu, Gaohong Liu, Jiawei Liu, Jie Liu, Jihao Liu, Shouda Liu, Shu Liu, Sichao Liu, Songwei Liu, Xin Liu, Xue Liu, Yibo Liu, Zikun Liu, Zuxi Liu, Junlin Lyu, Lecheng Lyu, Qian Lyu, Han Mu, Xiaonan Nie, Jingzhe Ning, Xitong Pan, Yanghua Peng, Lianke Qin, Xueqiong Qu, Yuxi Ren, Kai Shen, Guang Shi, Lei Shi, Yan Song, Yinglong Song, Fan Sun, Li Sun, Renfei Sun, Yan Sun, Zeyu Sun, Wenjing Tang, Yaxue Tang, Zirui Tao, Feng Wang, Furui Wang, Jinran Wang, Junkai Wang, Ke Wang, Kexin Wang, Qingyi Wang, Rui Wang, Sen Wang, Shuai Wang, Tingru Wang, Weichen Wang, Xin Wang, Yanhui Wang, Yue Wang, Yuping Wang, Yuxuan Wang, Ziyu Wang, Guoqiang Wei, Wanru Wei, Di Wu, Guohong Wu, Hanjie Wu, Jian Wu, Jie Wu, Ruolan Wu, Xinglong Wu, Yonghui Wu, Ruiqi Xia, Liang Xiang, Fei Xiao, XueFeng Xiao, Pan Xie, Shuangyi Xie, Shuang Xu, Jinlan Xue, Shen Yan, Bangbang Yang, Ceyuan Yang, Jiaqi Yang, Runkai Yang, Tao Yang, Yang Yang, Yihang Yang, ZhiXian Yang, Ziyan Yang, Songting Yao, Yifan Yao, Zilyu Ye, Bowen Yu, Jian Yu, Chujie Yuan, Linxiao Yuan, Sichun Zeng, Weihong Zeng, Xuejiao Zeng, Yan Zeng, Chuntao Zhang, Heng Zhang, Jingjie Zhang, Kuo Zhang, Liang Zhang, Liying Zhang, Manlin Zhang, Ting Zhang, Weida Zhang, Xiaohe Zhang, Xinyan Zhang, Yan Zhang, Yuan Zhang, Zixiang Zhang, Fengxuan Zhao, Huating Zhao, Yang Zhao, Hao Zheng, Jianbin Zheng, Xiaozheng Zheng, Yangyang Zheng, Yijie Zheng, Jiexin Zhou, Jiahui Zhu, Kuan Zhu, Shenhan Zhu, Wenjia Zhu, Benhui Zou, Feilong Zuo

专题命中 偏好对齐 ：RLHF(abstract)

AI总结 Seedance 1.5 pro是一款专为原生音频视频联合生成设计的基础模型，通过双分支扩散变压器架构实现高质量生成，支持多语言唇形同步和动态摄像机控制，提升专业内容创作效率。

Comments Seedance 1.5 pro Technical Report

Reinforcement Learning for Large Model: A Survey

为大模型设计强化学习：综述

Weijia Wu, Chen Gao, Joya Chen, Kevin Qinghong Lin, Qingwei Meng, Yiming Zhang, Yuke Qiu, Hong Zhou, Mike Zheng Shou

机构 * National University of Singapore（新加坡国立大学） ； Zhejiang University（浙江大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 偏好对齐 ：RLHF(abstract)

AI总结 本文综述了视觉强化学习领域，探讨了多模态大语言模型、视觉生成等四个主题，分析了算法设计、奖励工程及评估协议，并指出了样本效率、泛化能力等开放挑战。

Comments 22 pages

Emotion-Director: Bridging Affective Shortcut in Emotion-Oriented Image Generation

情绪导演：在情绪导向图像生成中弥合情感捷径

Guoli Jia, Junyao Hu, Xinwei Long, Kai Tian, Kaiyan Zhang, KaiKai Zhao, Ning Ding, Bowen Zhou

机构 * Tsinghua University（清华大学） ； The Hong Kong Polytechnic University（香港理工大学） ； China Unicom（中国联合通信） ； Shanghai Artificial Intelligence Lab（上海人工智能实验室） 纠错

专题命中 偏好对齐 ：DPO(abstract)

AI总结 Emotion-Director通过跨模态协作框架，结合MC-Diffusion和MC-Agent，提升情绪导向图像生成的准确性和表现力。

Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation

自奖励的大型视觉-语言模型用于优化文本到图像生成中的提示

Hongji Yang, Yucheng Zhou, Wencheng Han, Jianbing Shen

机构 * University of Macau（澳门大学） 纠错

专题命中 偏好对齐 ：alignment(abstract)

AI总结 本文提出了一种自奖励的大型视觉-语言模型框架，用于优化文本到图像生成中的提示，通过统一求解器和奖励模型实现自我改进，实验表明其优于其他方法。

Comments Accepted by ACL2025 Findings

LDP: Parameter-Efficient Fine-Tuning of Multimodal LLM for Medical Report Generation

LDP：多模态大语言模型在医疗报告生成中的参数高效微调

Tianyu Zhou, Junyi Tang, Zehui Li, Dahong Qian, Suncheng Xiang

专题命中 偏好对齐 ：DPO(abstract)

AI总结 LDP通过多模态大语言模型和参数高效微调技术，提升结肠镜息肉诊断报告的准确性和效率，显著降低训练成本并获得专家评分。

Comments Work in progress

Distilling Expert Surgical Knowledge: How to train local surgical VLMs for anatomy explanation in Complete Mesocolic Excision

萃取专家手术知识：如何训练局部手术VLMs用于完全网膜切除术的解剖解释

Lennart Maack, Julia-Kristin Graß, Lisa-Marie Toscha, Nathaniel Melling, Alexander Schlaefer

机构 * 1 Institute of Medical Technology ； Intelligent Systems, Hamburg University of Technology, Hamburg, Germany 2 Department of General, Visceral ； Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany 纠错

专题命中 偏好对齐 ：DPO(abstract)

AI总结 本文提出一种隐私保护框架，通过从大型通用LLM中萃取知识，训练出高效的本地手术VLM，用于在完全网膜切除术中进行解剖解释。

Self-Improving VLM Judges Without Human Annotations

无需人工标注的自改进VLM评判模型

Inna Wanyin Lin, Yushi Hu, Shuyue Stella Li, Scott Geng, Pang Wei Koh, Luke Zettlemoyer, Tim Althoff, Marjan Ghazvininejad

机构 * FAIR at Meta（Meta 的 FAIR） ； University of Washington（华盛顿大学） 纠错

专题命中 偏好对齐 ：safety(abstract)

AI总结 无需人工标注，通过自训练提升VLM评判模型的准确性和多维度表现

Multi-Reward GRPO for Stable and Prosodic Single-Codebook TTS LLMs at Scale

多奖励GRPO用于大规模稳定且有声调的单代码本TTS LLMs

Yicheng Zhong, Peiji Yang, Zhisheng Wang

机构 * Tencent Technology Co.Ltd（腾讯科技有限公司） 纠错

专题命中 偏好对齐 ：alignment(abstract)

AI总结 本文提出多奖励GRPO框架，通过优化单代码本TTS LLMs的生成策略，提升语音的语调稳定性、说话者相似性和自然度。

Comments 4 pages, 2 figures

UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learning

Rui Tian, Mingfei Gao, Haiming Gang, Jiasen Lu, Zhe Gan, Yinfei Yang, Zuxuan Wu, Afshin Dehghan

机构 * Institute of Trustworthy Embodied AI, Fudan University（可信具身人工智能研究院，复旦大学） ； Apple（苹果公司） 纠错

专题命中 偏好对齐 ：alignment(abstract)

Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localization

Tiancheng Yang, Lin Zhang, Jiaye Lin, Guimin Hu, Di Wang, Lijie Hu

机构 * MBZUAI ； Provable Responsible AI and Data Analytics (PRADA) Lab（可证明负责任的人工智能与数据分析实验室） ； King Abdullah University of Science and Technology（卡迪夫大学科学与技术大学） ； School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院） ； University of Copenhagen（哥本哈根大学） ； Tsinghua University（清华大学） 纠错

专题命中 偏好对齐 ：DPO(abstract)

SRLF: An Agent-Driven Set-Wise Reflective Learning Framework for Sequential Recommendation

Jiahao Wang, Bokang Fu, Yu Zhu, Yuli Liu

专题命中 偏好对齐 ：alignment(abstract)

PROPA: Toward Process-level Optimization in Visual Reasoning via Reinforcement Learning

Yanbei Jiang, Chao Lei, Yihao Ding, Krista Ehinger, Jey Han Lau

机构 * University of Melbourne（墨尔本大学） 纠错

专题命中 偏好对齐 ：alignment(abstract)

RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation

Yue Fang, Jin Zhi, Jie An, Hongshen Chen, Xiaohong Chen, Naijun Zhan

专题命中 偏好对齐 ：alignment(abstract)

Comments 12 pages, 4 figures

FlowMesh: A Service Fabric for Composable LLM Workflows

Junyi Shen, Noppanat Wadlom, Lingfeng Zhou, Dequan Wang, Xu Miao, Lei Fang, Yao Lu

专题命中 偏好对齐 ：RLHF(abstract)

From Time and Place to Preference: LLM-Driven Geo-Temporal Context in Recommendations

Yejin Kim, Shaghayegh Agah, Mayur Nankani, Neeraj Sharma, Feifei Peng, Maria Peifer, Sardar Hamidian, H Howie Huang

专题命中 偏好对齐 ：alignment(abstract)

OneRec-V2 Technical Report

Guorui Zhou, Hengrui Hu, Hongtao Cheng, Huanjie Wang, Jiaxin Deng, Jinghao Zhang, Kuo Cai, Lejian Ren, Lu Ren, Liao Yu, Pengfei Zheng, Qiang Luo, Qianqian Wang, Qigen Hu, Rui Huang, Ruiming Tang, Shiyao Wang, Shujie Yang, Tao Wu, Wuchao Li, Xinchen Luo, Xingmei Wang, Yi Su, Yunfan Wu, Zexuan Cheng, Zhanyu Liu, Zixing Zhang, Bin Zhang, Boxuan Wang, Chaoyi Ma, Chengru Song, Chenhui Wang, Chenglong Chu, Di Wang, Dongxue Meng, Dunju Zang, Fan Yang, Fangyu Zhang, Feng Jiang, Fuxing Zhang, Gang Wang, Guowang Zhang, Han Li, Honghui Bao, Hongyang Cao, Jiaming Huang, Jiapeng Chen, Jiaqiang Liu, Jinghui Jia, Kun Gai, Lantao Hu, Liang Zeng, Qiang Wang, Qidong Zhou, Rongzhou Zhang, Shengzhe Wang, Shihui He, Shuang Yang, Siyang Mao, Sui Huang, Tiantian He, Tingting Gao, Wei Yuan, Xiao Liang, Xiaoxiao Xu, Xugang Liu, Yan Wang, Yang Zhou, Yi Wang, Yiwu Liu, Yue Song, Yufei Zhang, Yunfeng Zhao, Zhixin Ling, Ziming Li

专题命中 偏好对齐 ：alignment(abstract)

Recognizing internal states in AI: evidence from patterned preferences in large language models

Annika Hedberg

专题命中 偏好对齐 ：alignment(abstract)

Comments 14 pages, 1 table. Includes test protocol transcript and multi-system response data

Generative Reasoning Recommendation via LLMs

Minjie Hong, Zetong Zhou, Zirun Guo, Ziang Zhang, Ruofan Hu, Weinan Gan, Jieming Zhu, Zhou Zhao

专题命中 偏好对齐 ：alignment(abstract)

Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator

Hualei Wang, Na Li, Chuke Wang, Shu Wu, Zhifeng Li, Dong Yu

专题命中 偏好对齐 ：alignment(abstract)

Comments 10 pages, 5 figures

Dual Caption Preference Optimization for Diffusion Models

Amir Saeidi, Yiran Luo, Agneet Chatterjee, Shamanthak Hegde, Bimsara Pathiraja, Yezhou Yang, Chitta Baral

机构 * School of Computing and Augmented Intelligence（计算与增强智能学院） ； Arizona State University（亚利桑那州立大学） 纠错

专题命中 偏好对齐 ：DPO(abstract)

Spatial Preference Rewarding for MLLMs Spatial Understanding

Han Qiu, Peng Gao, Lewei Lu, Xiaoqin Zhang, Ling Shao, Shijian Lu

机构 * S-Lab, Nanyang Technological University（南洋理工大学S实验室） ； Shanghai AI Laboratory（上海人工智能实验室） ； Sensetime Research（商汤科技研究院） ； Zhejiang University of Technology（浙江工业大学） ； UCAS-Terminus AI Lab,University of Chinese Academy of Sciences（中国科学院大学Terminus AI实验室） 纠错

专题命中 偏好对齐 ：alignment(abstract)

Comments ICCV 2025

Synergistic Integration and Discrepancy Resolution of Contextualized Knowledge for Personalized Recommendation

Lingyu Mu, Hao Deng, Haibo Xing, Kaican Lin, Zhitong Zhu, Yu Zhang, Xiaoyi Zeng, Zhengxiao Liu, Zheng Lin, Jinxin Hu

专题命中 偏好对齐 ：alignment(abstract)

Beyond Words: Infusing Conversational Agents with Human-like Typing Behaviors

Jijie Zhou, Yuhan Hu

专题命中 偏好对齐 ：trustworthy(abstract)

Comments Author's version of a paper published at CUI '24 (ACM Conversational User Interfaces 2024)

Journal ref CUI '24: Proceedings of the ACM Conversational User Interfaces 2024, July 8-10, 2024, Luxembourg, Luxembourg. ACM, New York, NY, USA, 11 pages

Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization

Tao Zhang, Cheng Da, Kun Ding, Huan Yang, Kun Jin, Yan Li, Tingting Gao, Di Zhang, Shiming Xiang, Chunhong Pan

机构 * MAIS, CASIA（中国科学院自动化研究所MAIS实验室） ； Kuaishou Technology（快手科技） ； School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院） 纠错

专题命中 偏好对齐 ：alignment(abstract)

Comments NeurIPS 2025

ImageDoctor: Diagnosing Text-to-Image Generation via Grounded Image Reasoning

Yuxiang Guo, Jiang Liu, Ze Wang, Hao Chen, Ximeng Sun, Yang Zhao, Jialian Wu, Xiaodong Yu, Zicheng Liu, Emad Barsoum

机构 * Johns Hopkins University（约翰霍普金斯大学） ； AMD（AMD公司） 纠错

专题命中 偏好对齐 ：alignment(abstract)

Follow-Your-Preference: Towards Preference-Aligned Image Inpainting

Yutao Shen, Junkun Yuan, Toru Aonishi, Hideki Nakayama, Yue Ma

机构 * The University of Tokyo（东京大学） ； Zhejiang University（浙江大学） ； Tsinghua University（清华大学） 纠错

专题命中 偏好对齐 ：alignment(abstract)

Comments 16 pages,9 figures, 4 tables

SeamCrafter: Enhancing Mesh Seam Generation for Artist UV Unwrapping via Reinforcement Learning

Duoteng Xu, Yuguang Chen, Jing Li, Xinhai Liu, Xueqi Ma, Zhuo Chen, Dongyu Zhang, Chunchao Guo

机构 * Tencent Hunyuan（腾讯文英） ； SYSU（南方科技大学） ； SZU（深圳大学） ； USTC（中国科学技术大学） 纠错

专题命中 偏好对齐 ：DPO(abstract)

Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation

Jihao Gu, Yingyao Wang, Meng Cao, Pi Bu, Jun Song, Yancheng He, Shilong Li, Bo Zheng

机构 * Taobao & Tmall Group of Alibaba（淘宝与天猫集团） 纠错

专题命中 偏好对齐 ：DPO(abstract)

Temporal-Aware User Behaviour Simulation with Large Language Models for Recommender Systems

Xinye Wanyan, Danula Hettiachchi, Chenglong Ma, Ziqi Xu, Jeffrey Chan

专题命中 偏好对齐 ：alignment(abstract)