大模型对齐与安全

RLHF Fine-Tuning of LLMs for Alignment with Implicit User Feedback in Conversational Recommenders

Zhongheng Yang, Aijia Sun, Yushang Zhao, Yinuo Yang, Dannier Li, Chengrui Zhou

专题命中 偏好对齐 ：alignment(title,abstract)；RLHF(title,abstract)；分类 cs.LG

SafeWork-R1: Coevolving Safety and Intelligence under the AI-45$^{\circ}$ Law

Shanghai AI Lab, :, Yicheng Bao, Guanxu Chen, Mingkang Chen, Yunhao Chen, Chiyu Chen, Lingjie Chen, Sirui Chen, Xinquan Chen, Jie Cheng, Yu Cheng, Dengke Deng, Yizhuo Ding, Dan Ding, Xiaoshan Ding, Yi Ding, Zhichen Dong, Lingxiao Du, Yuyu Fan, Xinshun Feng, Yanwei Fu, Yuxuan Gao, Ruijun Ge, Tianle Gu, Lujun Gui, Jiaxuan Guo, Qianxi He, Yuenan Hou, Xuhao Hu, Hong Huang, Kaichen Huang, Shiyang Huang, Yuxian Jiang, Shanzhe Lei, Jie Li, Lijun Li, Hao Li, Juncheng Li, Xiangtian Li, Yafu Li, Lingyu Li, Xueyan Li, Haotian Liang, Dongrui Liu, Qihua Liu, Zhixuan Liu, Bangwei Liu, Huacan Liu, Yuexiao Liu, Zongkai Liu, Chaochao Lu, Yudong Lu, Xiaoya Lu, Zhenghao Lu, Qitan Lv, Caoyuan Ma, Jiachen Ma, Xiaoya Ma, Zhongtian Ma, Lingyu Meng, Ziqi Miao, Yazhe Niu, Yuezhang Peng, Yuan Pu, Han Qi, Chen Qian, Xingge Qiao, Jingjing Qu, Jiashu Qu, Wanying Qu, Wenwen Qu, Xiaoye Qu, Qihan Ren, Qingnan Ren, Qingyu Ren, Jing Shao, Wenqi Shao, Shuai Shao, Dongxing Shi, Xin Song, Xinhao Song, Yan Teng, Xuan Tong, Yingchun Wang, Xuhong Wang, Shujie Wang, Xin Wang, Yige Wang, Yixu Wang, Yuanfu Wang, Futing Wang, Ruofan Wang, Wenjie Wang, Yajie Wang, Muhao Wei, Xiaoyu Wen, Fenghua Weng, Yuqi Wu, Yingtong Xiong, Xingcheng Xu, Chao Yang, Yue Yang, Yang Yao, Yulei Ye, Zhenyun Yin, Yi Yu, Bo Zhang, Qiaosheng Zhang, Jinxuan Zhang, Yexin Zhang, Yinqiang Zheng, Hefeng Zhou, Zhanhui Zhou, Pengyu Zhu, Qingzi Zhu, Yubo Zhu, Bowen Zhou

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 偏好对齐 ：safety(title,abstract)；alignment(abstract)；RLHF(abstract)；trustworthy(abstract)

Comments 47 pages, 18 figures, authors are listed in alphabetical order by their last names; v3 modifies minor issues

RLTHF: Targeted Human Feedback for LLM Alignment

Yifei Xu, Tusher Chakraborty, Emre Kıcıman, Bibek Aryal, Eduardo Rodrigues, Srinagesh Sharma, Roberto Estevao, Maria Angels de Luis Balaguer, Jessica Wolk, Rafael Padilha, Leonardo Nunes, Shobana Balakrishnan, Songwu Lu, Ranveer Chandra

机构 * Microsoft（微软公司） ； University of California, Los Angeles（加州大学洛杉矶分校） 纠错

专题命中 偏好对齐 ：alignment(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Presented at ICML 2025

Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models

Mason Nakamura, Saaduddin Mahmud, Kyle H. Wray, Hamed Zamani, Shlomo Zilberstein

专题命中 偏好对齐 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Tae Soo Kim, Yoonjoo Lee, Yoonah Park, Jiho Kim, Young-Ho Kim, Juho Kim

机构 * KAIST（韩国科学技术院） ； Seoul National University（首尔国立大学） ； Calvin University（凯尔文大学） ； NAVER AI LAB（NAVER AI实验室） 纠错

专题命中 偏好对齐 ：alignment(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to COLM 2025. Project Website: https://cupid.kixlab.org/

A Metric for MLLM Alignment in Large-scale Recommendation

Yubin Zhang, Yanhua Huang, Haiming Xu, Mingliang Qi, Chang Wang, Jiarui Jin, Xiangyuan Ren, Xiaodan Wang, Ruiwen Xu

机构 * Xiaohongshu Inc.（小红书公司） 纠错

专题命中 偏好对齐 ：alignment(title,abstract)；分类 cs.LG

Comments Pre-print.Under Review

Towards Hallucination-Free Music: A Reinforcement Learning Preference Optimization Framework for Reliable Song Generation

Huaicheng Zhang, Wei Tan, Guangzheng Li, Yixuan Zhang, Hangting Chen, Shun Lei, Chenyu Yang, Zhiyong Wu, Shuai Wang, Qijun Huang, Dong Yu

机构 * Tencent AI Lab（腾讯AI实验室） 纠错

专题命中 偏好对齐 ：alignment(abstract)；DPO(abstract)；分类 cs.AI

GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

Jianghangfan Zhang, Yibo Yan, Kening Zheng, Xin Zou, Song Dai, Xuming Hu

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL

Building Effective Safety Guardrails in AI Education Tools

Hannah-Beth Clark, Laura Benton, Emma Searle, Margaux Dowland, Matthew Gregory, Will Gayne, John Roberts

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.CY

Comments 9 pages, published in proceedings of International Conference on Artificial Intelligence in Education (AIED) 2025: Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium, Blue Sky, and WideAIED

Journal ref Communications in Computer and Information Science (2025) vol 2590, pp. 129-136

PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages

Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, Maarten Sap

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL

Comments Accepted to COLM 2025 Main Conference

AI vs. Human Moderators: A Comparative Evaluation of Multimodal LLMs in Content Moderation for Brand Safety

Adi Levi, Or Levi, Sardhendu Mishra, Jonathan Morra

机构 * Zefr Inc（Zefr公司） 纠错

专题命中 安全训练 ：safety(title,abstract)

Comments Accepted to the Computer Vision in Advertising and Marketing (CVAM) workshop at ICCV 2025

Are Large Language Models Dynamic Treatment Planners? An In Silico Study from a Prior Knowledge Injection Angle

Zhiyao Luo, Tingting Zhu

机构 * Department of Engineering Science University of Oxford（工程科学系牛津大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.LG

Comments 20 pages

JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering

Renmiao Chen, Shiyao Cui, Xuancheng Huang, Chengwei Pan, Victor Shea-Jay Huang, QingLin Zhang, Xuan Ouyang, Zhexin Zhang, Hongning Wang, Minlie Huang

机构 * CoAI group, DCST, Tsinghua University（清华大学DCST学院） ； Beihang University（北航大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 3 tables, 2 figures, to appear in the Proceedings of the 33rd ACM International Conference on Multimedia (MM '25)

IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards

Xu Guo, Tianyi Liang, Tong Jian, Xiaogui Yang, Ling-I Wu, Chenhui Li, Zhihui Lu, Qipeng Guo, Kai Chen

专题命中 越狱攻击 ：alignment(abstract)；分类 cs.CL

Comments 7 pages, 4 figures

Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling

Jianpeng Yao, Xiaopan Zhang, Yu Xia, Zejin Wang, Amit K. Roy-Chowdhury, Jiachen Li

机构 * University of California, Riverside（加州大学河滨分校） 纠错

专题命中 幻觉与事实性 ：safety(title)；分类 cs.AI、cs.LG

Comments 9th Conference on Robot Learning (CoRL 2025); Project website: https://gen-safe-nav.github.io/. arXiv admin note: text overlap with arXiv:2407.17460

$\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection

Daniil Orel, Indraneil Paul, Iryna Gurevych, Preslav Nakov

机构 * Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（Mohamed bin Zayed人工智能大学） ； Ubiquitous Knowledge Processing Lab (UKP Lab)（普及知识处理实验室） ； Department of Computer Science（计算机科学系） ； National Research Center for Applied Cybersecurity ATHENE（应用网络安全国家研究中心ATHENE） 纠错

专题命中 幻觉与事实性 ：alignment(abstract)；分类 cs.AI、cs.CY

GrandJury: A Collaborative Machine Learning Model Evaluation Protocol for Dynamic Quality Rubrics

Arthur Cho

机构 * Memoirji LLC（Memiorji公司） 纠错

专题命中 安全评测 ：alignment(abstract)；分类 cs.AI、cs.LG

Comments 14 pages (incl. arXiv cover), 1 table, code & dataset links inside. Open-source implementation available on PyPI (grandjury package) and GitHub. Dataset available on Hugging Face under CC-BY-4.0 license

Auto-Eval Judge: Towards a General Agentic Framework for Task Completion Evaluation

Roshita Bhonsle, Rishav Dutta, Sneha Vavilapalli, Harsh Seth, Abubakarr Jaye, Yapei Chang, Mukund Rungta, Emmanuel Aboah Boateng, Sadid Hasan, Ehi Nosakhare, Soundar Srinivasan

机构 * University of Massachusetts Amherst（马萨诸塞大学阿姆赫斯特分校） ； Microsoft Corporation（微软公司） ； University of Maryland（马里兰大学） 纠错

专题命中 安全评测 ：alignment(abstract)；分类 cs.AI

TASE: Token Awareness and Structured Evaluation for Multilingual Language Models

Chenzhuo Zhao, Xinda Wang, Yue Huang, Junting Lu, Ziqian Liu

专题命中 安全评测 ：alignment(abstract)；分类 cs.CL

Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models

Zane Xu, Jason Sun

专题命中 安全评测 ：alignment(abstract)；分类 cs.AI

MedMKEB: A Comprehensive Knowledge Editing Benchmark for Medical Multimodal Large Language Models

Dexuan Xu, Jieyi Wang, Zhongyan Chai, Yongzhi Cao, Hanpin Wang, Huamin Zhang, Yu Huang

专题命中 安全评测 ：trustworthy(abstract)；分类 cs.AI

Comments 18 pages

Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity

Yuhan Zhang, Long Zhuo, Ziyang Chu, Tong Wu, Zhibing Li, Liang Pan, Dahua Lin, Ziwei Liu

机构 * Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Tsinghua University（清华大学） ； Stanford University（斯坦福大学） ； The Chinese University of Hong Kong（香港中文大学） ； S-Lab, Nanyang Technological University（南洋理工大学S实验室） 纠错

专题命中 安全评测 ：alignment(abstract)

Comments Page: https://zyh482.github.io/Hi3DEval/

From Detection to Correction: Backdoor-Resilient Face Recognition via Vision-Language Trigger Detection and Noise-Based Neutralization

Farah Wahida, M. A. P. Chamikara, Yashothara Shanmugarasa, Mohan Baruwal Chhetri, Thilina Ranbaduge, Ibrahim Khalil

机构 * RMIT University, Australia（皇家墨尔本理工大学） 纠错

专题命中 安全评测 ：trustworthy(abstract)

Comments 19 Pages, 24 Figures

Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI

Krzysztof Janowicz, Zilong Liu, Gengchen Mai, Zhangyu Wang, Ivan Majic, Alexandra Fortacz, Grant McKenzie, Song Gao

机构 * University of Vienna（维也纳大学） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； University of Maine（缅因大学） ； McGill University（麦吉尔大学） ； University of Wisconsin（威斯康星大学） 纠错

专题命中 AI治理与伦理 ：alignment(title,abstract)；分类 cs.AI、cs.CY

Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives

Wei Zeng, Hengshu Zhu, Chuan Qin, Han Wu, Yihang Cheng, Sirui Zhang, Xiaowei Jin, Yinuo Shen, Zhenxing Wang, Feimin Zhong, Hui Xiong

机构 * Business School, Hunan University（湖南大学商学院） ； Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心） ； University of Chinese Academy of Sciences（中国科学院大学） ； School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院） ； School of Business, Hunan University（湖南大学商学院） ； Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）人工智能研究所） ； Department of Computer Science and Engineering, The Hong Kong University of Science and Technology Hong Kong SAR（香港科技大学（香港）计算机科学与工程学院） 纠错

专题命中 AI治理与伦理 ：alignment(title,abstract)；分类 cs.AI

TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolution

Zhikai Zhao, Chuanbo Hua, Federico Berto, Kanghoon Lee, Zihan Ma, Jiachen Li, Jinkyoo Park

专题命中 其他安全 ：safety(abstract)；分类 cs.AI、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2505.04480

UNCAGE: Contrastive Attention Guidance for Masked Generative Transformers in Text-to-Image Generation

Wonjun Kang, Byeongkeun Ahn, Minjae Lee, Kevin Galim, Seunghyuk Oh, Hyung Il Koo, Nam Ik Cho

专题命中 其他安全 ：alignment(abstract)；分类 cs.AI、cs.LG

Comments Code is available at https://github.com/furiosa-ai/uncage

Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes

Sadia Kamal, Tim Oates, Joy Wan

机构 * Department of Computer Science, University of Maryland, Baltimore County（计算机科学系，马里兰大学巴尔的摩县分校） ； Department of Dermatology, Johns Hopkins University School of Medicine（皮肤科系，约翰霍普金斯大学医学院） 纠错

专题命中 其他安全 ：alignment(abstract)；分类 cs.AI、cs.LG

Comments Accepted to IJCAI 2025 Workshops. arXiv admin note: substantial text overlap with arXiv:2506.10328

Efficient Knowledge Injection in LLMs via Self-Distillation

Kalle Kujanpää, Pekka Marttinen, Harri Valpola, Alexander Ilin

机构 * Aalto University（阿alto大学） ； Finnish Center for Artificial Intelligence (FCAI)（芬兰人工智能中心） ； System 2 AI 纠错

专题命中 其他安全 ：alignment(abstract)；分类 cs.CL、cs.LG

Comments Preprint

Large Language Models Transform Organic Synthesis From Reaction Prediction to Automation

Kartar Kumar Lohana Tharwani, Rajesh Kumar, Sumita, Numan Ahmed, Yong Tang

机构 * International Research Center for Complexity Sciences, Hangzhou International Innovation Institute, Beihang University（国际复杂科学研究中心，杭州创新研究院，北京航空航天大学） ； School of Computer Science and Engineering, University of Electronic Science and Technology of China（计算机科学与工程学院，电子科学与技术大学） ； Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China（长江三角洲地区研究所（湖州），电子科学与技术大学） ； Government Boys Higher Secondary School, Bukera Sharif, Tando Allahyar, Affiliated with BISE Hyderabad, Sindh, Pakistan（政府男生高级中学，布克里·沙里夫，塔ndo阿勒·耶尔，隶属于Hyderabad BISE，Sindh，巴基斯坦） ； School of Environment and Architecture, University of Shanghai for Science and Technology（环境与建筑学院，上海科学技术大学） 纠错

专题命中 其他安全 ：safety(abstract)；分类 cs.AI