大模型对齐与安全

MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models

Chejian Xu, Jiawei Zhang, Zhaorun Chen, Chulin Xie, Mintong Kang, Yujin Potter, Zhun Wang, Zhuowen Yuan, Alexander Xiong, Zidi Xiong, Chenhui Zhang, Lingzhi Yuan, Yi Zeng, Peiyang Xu, Chengquan Guo, Andy Zhou, Jeffrey Ziwei Tan, Xuandong Zhao, Francesco Pinto, Zhen Xiang, Yu Gai, Zinan Lin, Dan Hendrycks, Bo Li, Dawn Song

专题命中 安全评测 ：safety(title,abstract)；red teaming(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025

A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety

Rakeen Rouf, Trupti Bavalatti, Osama Ahmed, Dhaval Potdar, Faraz Jawed

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication in IEEE Access, DOI: 10.1109/ACCESS.2025.3539933

Journal ref IEEE Access 2025

Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning

Ang Li, Yichuan Mo, Mingjie Li, Yifei Wang, Yisen Wang

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI

Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey

Bo Ni, Zheyuan Liu, Leyao Wang, Yongjia Lei, Yuying Zhao, Xueqi Cheng, Qingkai Zeng, Luna Dong, Yinglong Xia, Krishnaram Kenthapadi, Ryan Rossi, Franck Dernoncourt, Md Mehrab Tanjim, Nesreen Ahmed, Xiaorui Liu, Wenqi Fan, Erik Blasch, Yu Wang, Meng Jiang, Tyler Derr

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, Chaowei Xiao

专题命中 安全评测 ：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL、cs.AI

Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees

Yu Gui, Ying Jin, Zhimei Ren

专题命中 安全评测 ：alignment(title,abstract)；trustworthy(abstract)；分类 cs.AI、cs.LG

Alignment for Honesty

Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, Pengfei Liu

专题命中 安全评测 ：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework

Fan Liu, Yue Feng, Zhao Xu, Lixin Su, Xinyu Ma, Dawei Yin, Hao Liu

专题命中 安全评测 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents

Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, Rui Wang, Gongshen Liu

专题命中 安全评测 ：safety(title,abstract)；harmlessness(abstract)；分类 cs.CL、cs.AI

Comments EMNLP Findings 2024

Enabling Trustworthy Federated Learning in Industrial IoT: Bridging the Gap Between Interpretability and Robustness

Senthil Kumar Jagatheesaperumal, Mohamed Rahouti, Ali Alfatemi, Nasir Ghani, Vu Khanh Quy, Abdellah Chehri

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.LG

Comments 7 pages, 2 figures

Journal ref IEEE Internet of Things Magazine, Year: 2024, Volume: 7, Issue: 5

Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Framework

Matthew Pisano, Peter Ly, Abraham Sanders, Bingsheng Yao, Dakuo Wang, Tomek Strzalkowski, Mei Si

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging

Karim Lekadir, Richard Osuala, Catherine Gallin, Noussair Lazrak, Kaisar Kushibar, Gianna Tsakou, Susanna Aussó, Leonor Cerdá Alberich, Kostas Marias, Manolis Tsiknakis, Sara Colantonio, Nickolas Papanikolaou, Zohaib Salahuddin, Henry C Woodruff, Philippe Lambin, Luis Martí-Bonmatí

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.LG

Comments Please refer to arXiv:2309.12325 for the latest FUTURE-AI framework for healthcare

A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability

Ting Fang Tan, Kabilan Elangovan, Jasmine Ong, Nigam Shah, Joseph Sung, Tien Yin Wong, Lan Xue, Nan Liu, Haibo Wang, Chang Fu Kuo, Simon Chesterman, Zee Kin Yeong, Daniel SW Ting

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI

PRISM: A Design Framework for Open-Source Foundation Model Safety

Terrence Neumann, Bryan Jones

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI、cs.CY

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Tianle Gu, Zeyang Zhou, Kexin Huang, Dandan Liang, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Xingge Qiao, Keqing Wang, Yujiu Yang, Yan Teng, Yu Qiao, Yingchun Wang

专题命中 安全评测 ：safety(title,abstract)；red teaming(abstract)；分类 cs.CL、cs.AI

A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians

Piotr Wojciech Mirowski, Juliette Love, Kory W. Mathewson, Shakir Mohamed

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 1 figure, published at ACM FAccT 2024

How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities

Lingbo Mo, Boshi Wang, Muhao Chen, Huan Sun

专题命中 安全评测 ：trustworthy(title)；alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2024

Secure and Trustworthy Artificial Intelligence-Extended Reality (AI-XR) for Metaverses

Adnan Qayyum, Muhammad Atif Butt, Hassan Ali, Muhammad Usman, Osama Halabi, Ala Al-Fuqaha, Qammer H. Abbasi, Muhammad Ali Imran, Junaid Qadir

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.CY

Comments 24 pages, 11 figures

Journal ref ACM Computing Surveys (2023)

A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attacks

Yanjie Li, Bin Xie, Songtao Guo, Yuanyuan Yang, Bin Xiao

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.AI、cs.LG

Comments Submitted to CSUR

A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, Andre Freitas, Mustafa A. Mustafa

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI、cs.LG

Finding differences in perspectives between designers and engineers to develop trustworthy AI for autonomous cars

Gustav Jonelid, K. R. Larsson

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.CY

Survey of Trustworthy AI: A Meta Decision of AI

Caesar Wu, Yuan-Fang Lib, Pascal Bouvry

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.CY

A Comprehensive Survey on Trustworthy Recommender Systems

Wenqi Fan, Xiangyu Zhao, Xiao Chen, Jingran Su, Jingtong Gao, Lin Wang, Qidong Liu, Yiqi Wang, Han Xu, Lei Chen, Qing Li

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI、cs.LG

Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

脚手架下的安全性：评估条件如何影响测量的安全性

David Gringras

机构 * Harvard University（哈佛大学） ； MIT（麻省理工学院） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

AI总结 本研究通过62,808次盲法预注册评估，测试了六种前沿模型在四种部署配置下的安全性，发现脚手架架构对安全性影响较小，而格式转换（如选择题与开放式问题）可导致5-20个百分点的测量差异，且模型-脚手架间存在显著异质性，质疑了单一综合安全性分数的实用性。

Comments 74 pages including appendices. 6 frontier models, 62,808 primary observations (~89k total). Pre-registered: OSF DOI 10.17605/OSF.IO/CJW92. Code and data: https://github.com/davidgringras/safety-under-scaffolding

ThaiSafetyBench: Assessing Language Model Safety in Thai Cultural Contexts

ThaiSafetyBench: 评估泰语文化背景下语言模型的安全性

Trapoom Ukarapol, Nut Chukamphaeng, Kunat Pipatanakul, Pakhapoom Sarapat

机构 * SCB DataX（SCB数据X） ； Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系） ； SCBX R&D（SCBX研发部） ； SCB 10X 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL；trustworthy(comments)

AI总结 本研究提出ThaiSafetyBench，通过泰语文化背景下的恶意提示评估语言模型安全性，发现闭源模型安全性更强，同时揭示了文化特定攻击的高风险。

Comments ICLR 2026 Workshop on Principled Design for Trustworthy AI

From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0

从隐私到信任在代理时代：通过信任报告2.0的视角，对可信联邦学习中挑战的分类

Nuria Rodríguez-Barroso, Mario García-Márquez, M. Victoria Luzón, Francisco Herrera

机构 * Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada（计算机科学与人工智能系，数据科学与计算智能安达卢西亚研究 institute，格拉纳达大学） ； Department of Software Engineering, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada（软件工程系，数据科学与计算智能安达卢西亚研究 institute，格拉纳达大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.AI

AI总结 本文通过信任报告2.0提出可信联邦学习的挑战分类，强调信任作为持续维持的操作条件，并引入协调蓝图以处理跨要求的权衡和治理对齐。

Comments Already published in Information Fusion

Journal ref Rodríguez-Barroso, et. al. (2026). From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0. Information Fusion, 104236

AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systems

AEMA：可验证评估框架用于可信和受控的代理LLM系统

YenTing Lee, Keerthi Koneru, Zahra Moslemi, Sheethal Kumar, Ramesh Radhakrishnan

机构 * University of California, San Diego（加州大学圣地亚哥分校） ； Center for Advanced AI, Accenture（Accenture高级人工智能中心） ； University of California, Irvine（加州大学伊拉斯姆斯分校） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract,comments)；分类 cs.AI

AI总结 AEMA提出了一种可验证的评估框架，用于评估基于LLM的多代理系统，通过人类监督实现稳定、可追溯的自动化评估。

Comments Workshop on W51: How Can We Trust and Control Agentic AI? Toward Alignment, Robustness, and Verifiability in Autonomous LLM Agents at AAAI 2026

GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods

Ruixuan Huang, Xunguang Wang, Zongjie Li, Daoyuan Wu, Shuai Wang

专题命中 安全评测 ：jailbreak(title,abstract)；safety(abstract,comments)；分类 cs.CL

Comments Homepage: https://sproutnan.github.io/AI-Safety_Benchmark/

EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settings

EG-ARSA：适用于低资源场景的基于专家知识的开放视觉道路安全审计模型

Md Thamed Bin Zaman Chowdhury, Moazzem Hossain

机构 * Bangladesh University of Engineering and Technology (BUET)（孟加拉工程技术大学（BUET）） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

AI总结 针对中低收入国家道路安全审计的资源限制，提出EGD框架，构建BD-ARSA数据集和EG-ARSA模型，实验显示其性能优于310亿参数教师模型及Gemini-2.5-Flash，为低资源场景道路安全审计提供有效方案。

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models

Omni-SafetyBench：视听大语言模型安全评估基准

Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Aiwei Liu, Lijie Wen

机构 * Tsinghua University（清华大学） ； Tongyi Lab（通义实验室） ； Peking University（北京大学） ； OpenRL Lab（OpenRL实验室） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

AI总结 研究针对现有OLLM安全基准的不足，构建Omni-SafetyBench基准，提出定制化评估指标，发现多数OLLM存在安全漏洞，为该领域研究提供重要支撑。

Comments ACM MM 2026 (Oral)