大模型对齐与安全

Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration

Aetheria：基于多智能体辩论与协作的多模态可解释内容安全框架

Yuxiang He, Jian Zhao, Yuchen Yuan, Tianle Zhang, Wei Cai, Haojie Cheng, Ziyan Shi, Ming Zhu, Haichuan Tang, Chi Zhang, Xuelong Li

机构 * Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信） ； Sichuan University（四川大学） ； Peking University（北京大学） ； Beijing Jiaotong University（北京交通大学） ； Harbin Institute of Technology（哈尔滨工业大学） ； China Railway Rolling Stock Corporation Limited（中国铁路滚动股票有限公司） 纠错

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.AI

AI总结 Aetheria通过多智能体辩论与协作机制，实现多模态内容安全的可解释性与高准确性，提升可信AI审查水平。

Comments https://github.com/Herrieson/Aetheria

DialogGuard: Multi-Agent Psychosocial Safety Evaluation of Sensitive LLM Responses

DialogGuard: 多智能体心理社会安全评估框架用于敏感LLM响应

Han Luo, Guy Laban

机构 * University of Leeds（利兹大学） ； Southwest Jiaotong University（西南交通大学） ； Department of Industrial Engineering and Management（工业工程与管理系） ； Ben-Gurion University of the Negev（贝内尔·加隆大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

AI总结 DialogGuard通过多智能体框架评估LLM响应的心理社会风险，提供高准确性的风险检测与可解释的评估结果，支持安全提示设计和脆弱用户应用的监督。

Leveraging Evidence-Guided LLMs to Enhance Trustworthy Depression Diagnosis

利用证据引导的LLM提升可信的抑郁症诊断

Yining Yuan, J. Ben Tamo, Micky C. Nnamdi, Yifei Wang, May D. Wang

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.AI

AI总结 本文提出EGDR框架，通过证据引导推理和置信度评分提升抑郁症诊断的准确性和可信度。

DeepKnown-Guard: A Proprietary Model-Based Safety Response Framework for AI Agents

Qi Li, Jianjun Xu, Pingtao Wei, Jiu Li, Peiqiang Zhao, Jiwei Shi, Xuan Zhang, Yanhui Yang, Xiaodong Hui, Peng Xu, Wenqin Shao

机构 * Beijing Caizhi Tech（北京彩智科技） 纠错

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.AI

Rethinking Deep Alignment Through The Lens Of Incomplete Learning

Thong Bach, Dung Nguyen, Thao Minh Le, Truyen Tran

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.LG

Comments AAAI'26

How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity

Zihan Ma, Dongsheng Zhu, Shudong Liu, Taolin Zhang, Junnan Liu, Qingqiu Li, Minnan Luo, Songyang Zhang, Kai Chen

机构 * School of Computer Science and Technology Xi’an Jiaotong University China（西安交通大学计算机科学与技术学院） ； Shanghai AI Laboratory（上海人工智能实验室） ； MOE KLINNS Lab Xi’an Jiaotong University China（西安交通大学教育部KLINNS实验室） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

RxSafeBench: Identifying Medication Safety Issues of Large Language Models in Simulated Consultation

Jiahao Zhao, Luxin Xu, Minghuan Tan, Lichao Zhang, Ahmadreza Argha, Hamid Alinejad-Rokny, Min Yang

机构 * Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院） ； University of Electronic Science and Technology of China（电子科技大学） ； Shenzhen University of Advanced Technology（深圳大学） ； School of Biomedical Engineering, UNSW Sydney（生物医学工程学院，UNSW悉尼） 纠错

专题命中 安全评测 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.AI

Comments To appear in BIBM2025

Trustworthy Quantum Machine Learning: A Roadmap for Reliability, Robustness, and Security in the NISQ Era

Ferhat Ozgur Catak, Jungwon Seo, Umit Cali

机构 * Department of Electrical Engineering and Computer Science, University of Stavanger, Norway（斯瓦尔巴大学电气工程与计算机科学系） ； School of Physics, Engineering and Technology, University of York, UK（约克大学物理、工程与技术学院） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI

Comments 22 Pages

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents

Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce, Zico Kolter, Nicolas Flammarion, Maksym Andriushchenko

机构 * EPFL（苏黎世联邦理工学院） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；prompt injection(abstract)；分类 cs.LG

Comments NeurIPS 2025 Datasets & Benchmarks Track (Spotlight)

Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models

Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma, Ananth Shreekumar, Jonathan Rosenthal, Muslum Ozgur Ozmen, Mikhail Kuznetsov, Z. Berkay Celik

机构 * Purdue University（普渡大学） ； Arizona State University（亚利桑那州立大学） ； Amazon（亚马逊） 纠错

专题命中 安全评测 ：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CL

SAGE: A Generic Framework for LLM Safety Evaluation

Madhur Jindal, Hari Shrawgi, Parag Agrawal, Sandipan Dandapat

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Comments Accepted to EMNLP 2025

Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety

Yubin Kim, Hyewon Jeong, Chanwoo Park, Eugene Park, Haipeng Zhang, Xin Liu, Hyeonhoon Lee, Daniel McDuff, Marzyeh Ghassemi, Cynthia Breazeal, Samir Tulebaev, Hae Won Park

机构 * Massachusetts Institute of Technology（麻省理工学院） ； Google Research（谷歌研究） ； Harvard Medical School（哈佛医学院） ； Seoul National University Hospital（首尔国立大学医院） 纠错

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential

Charankumar Akiri, Harrison Simpson, Kshitiz Aryal, Aarav Khanna, Maanak Gupta

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CY

AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models

Sayantan Adak, Pratyush Chatterjee, Somnath Banerjee, Rima Hazra, Somak Aditya, Animesh Mukherjee

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.CL

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Wanying Wang, Zeyu Ma, Han Zheng, Xin Tan, Mingang Chen

机构 * Shanghai Key Laboratory of Computer Software Testing and Evaluating（上海软件测试与评估 key laboratory） ； Shanghai Normal University（上海Normal University） ； TrustAI Pte. Ltd. ； East China Normal University（东华师范大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Comments Accepted by ACM Multimedia 2025

ELITE: Enhanced Language-Image Toxicity Evaluation for Safety

Wonjun Lee, Doehyeon Lee, Eugene Choi, Sangyoon Yu, Ashkan Yousefpour, Haon Park, Bumsub Ham, Suhyun Kim

机构 * Yonsei University（延世大学） ； Kyung Hee University（庆熙大学） ； Korea Institute of Science（韩国科学研究院） ； Seoul National University（首尔国立大学） ； Sookmyung Women's University（_sookmyung女子大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments ICML 2025. Project page at https://velpegor.github.io/ELITE/

Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework

Rishane Dassanayake, Mario Demetroudi, James Walpole, Lindley Lentati, Jason R. Brown, Edward James Young

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Comments 24 pages (14 pages main text, 4 pages bibliography, 6 pages appendices), 3 figures

CALMA: A Process for Deriving Context-aligned Axes for Language Model Alignment

Prajna Soni, Deepika Raman, Dylan Hadfield-Menell

专题命中 安全评测 ：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CY

Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework

Satyapriya Krishna, Ninareh Mehrabi, Abhinav Mohanty, Matteo Memelli, Vincent Ponzo, Payal Motwani, Rahul Gupta

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CY

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Artyom Kharinaev, Viktor Moskvoretskii, Egor Shvetsov, Kseniia Studenikina, Bykov Mikhail, Evgeny Burnaev

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency

Kathleen C. Fraser, Hillary Dawkins, Isar Nejadgholi, Svetlana Kiritchenko

机构 * National Research Council Canada（加拿大国家研究理事会） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments to appear at LLMSEC 2025

From Theory to Practice: Real-World Use Cases on Trustworthy LLM-Driven Process Modeling, Prediction and Automation

Peter Pfeiffer, Alexander Rombach, Maxim Majlatow, Nijat Mehdiyev

机构 * German Research Center for Artificial Intelligence (DFKI) and Saarland University（德国人工智能研究中心（DFKI）和萨尔兰州大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.LG

Comments Accepted to the Next Gen Data and Process Management: Large Language Models and Beyond workshop at SIGMOD 2025

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap

Wenhan Yang, Spencer Stice, Ali Payani, Baharan Mirzasoleiman

机构 * Computer Science Department UCLA（计算机科学系，加州大学洛杉矶分校） ； Cisco Systems Inc.（思科系统公司） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It

Zheng-Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen H. Bach, Julia Kreutzer

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CL

Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models

Hao Yang, Lizhen Qu, Ehsan Shareghi, Gholamreza Haffari

机构 * Department of Data Science & AI, Monash University（数据科学与人工智能系，莫纳什大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Mitigating Deceptive Alignment via Self-Monitoring

Jiaming Ji, Wenqi Chen, Kaile Wang, Donghai Hong, Sitong Fang, Boyuan Chen, Jiayi Zhou, Juntao Dai, Sirui Han, Yike Guo, Yaodong Yang

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.AI

From nuclear safety to LLM security: Applying non-probabilistic risk management strategies to build safe and secure LLM-powered systems

Alexander Gutfraind, Vicki Bier

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration

Honglong Yang, Shanshan Song, Yi Qin, Lehan Wang, Haonan Wang, Xinpeng Ding, Qixiang Zhang, Bodong Du, Xiaomeng Li

机构 * Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学） ； Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.CL

aiXamine: Simplified LLM Safety and Security

Fatih Deniz, Dorde Popovic, Yazan Boshmaf, Euisuh Jeong, Minhaj Ahmad, Sanjay Chawla, Issa Khalil

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation

Jiajun Shen, Tong Zhou, Yubo Chen, Delai Qiu, Shengping Liu, Kang Liu, Jun Zhao

机构 * The Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； University of Chinese Academy of Sciences（中国科学院大学） ； Unisound Al Technology Co,Ltd（Unisound人工智能技术有限公司） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.CL

Comments 19 pages, 14 figures