Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds across turns and input modalities. We present MUSE (Multimodal Unified Safety Evaluation), an open-source, browser-based, run-centric platform for multimodal safety evaluation. MUSE treats each attack run as the persistent unit of execution, inspection, and analysis, preserving its configuration, multi-turn trajectory, delivered modalities and media, target responses, and safety judgments. A five-level response taxonomy further distinguishes full Compliance from Partial Compliance and refusal behavior, yielding hard ASR, soft ASR, and gray-zone width (GZW).
Across 11,700 evaluations on six multimodal LLMs, direct text-only requests yield only 3.1% macro hard ASR and 4.4% soft ASR, while iterative attack procedures are substantially more effective. Attack effectiveness also varies substantially with the attacker backbone. In contrast, Inter-Turn Modality Switching (ITMS), evaluated as a controlled delivery-modality probe, does not consistently increase attack success. These results demonstrate the value of run-centric, fine-grained evaluation for characterizing multimodal safety behavior beyond a single binary success metric.
A Granular Study of Safety Pretraining under Model Abliteration
Data and Web Science Group, University of Mannheim(曼海姆大学数据与网络科学组);Vision and AI Lab, Indian Institute of Science(印度科学院视觉与人工智能实验室);Carnegie Mellon University(卡内基梅隆大学);Max-Planck-Institute for Informatics, Saarland Informatics Campus(马克斯·普朗克信息研究所,萨尔兰信息校园)
Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
Department of Informatics, University of Sussex(信息学院,苏塞克斯大学);Pingtan Research Institute, Xiamen University(平潭研究院,厦门大学);Department of Computer Science, University of Liverpool(计算机科学系,利物浦大学);Department of Computer Science, Purdue University(计算机科学系,普渡大学);Department of Computer Science, Emory University(计算机科学系,埃默里大学);AI Agent Lab, Vokram Group(AI代理实验室,Vokram集团);Department of Computing, Imperial College London(计算系,帝国理工学院伦敦分校)
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Renmin University of China(中国人民大学);Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Chinese University of Hong Kong(香港中文大学);Johns Hopkins University(约翰霍普金斯大学);Hong Kong University of Science and Technology(香港科技大学);Tencent(腾讯)
JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models
Kyoto University(京都大学);Hohai University(河海大学);The University of Tokyo(东京大学);University of Science and Technology of China(中国科学技术大学);Hong Kong Polytechnic University(香港理工大学)
From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich(自主车辆系统教授职位,技术大学慕尼黑工程与设计学院);Munich Institute of Robotics and Machine Intelligence (MIRMI)(慕尼黑机器人与机器智能研究所);TUM School of Computation, Information and Technology, Department of Computer Engineering, Technical University of Munich(技术大学慕尼黑计算、信息与技术学院,计算机工程系)
EC-IoU: Orienting Safety for Object Detectors via Ego-Centric Intersection-over-Union
DENSO AUTOMOTIVE Deutschland GmbH(电装德国汽车有限公司);Technical University of Munich(慕尼黑工业大学);Chalmers University of Technology(查尔姆斯理工大学);University of Gothenburg(哥德堡大学)
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
Google(谷歌);Oxford University(牛津大学);Harvard University(哈佛大学);CMU(卡内基梅隆大学);Cohere(Cohere公司);UCL(伦敦大学学院);University of Zurich(苏黎世大学);MLCommons(机器学习通用组织);Kaggle(Kaggle平台)
A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safety
Department of Computer Science & Engineering, Texas A&M University, College Station, USA(德克萨斯A&M大学计算机科学与工程系);Department of Mathematics, The University of Iowa, Iowa City, USA(爱荷华大学数学系);Global Research and Development, General Motors, Warren, USA(通用汽车全球研发);Department of Electrical and Computer Engineering, The Ohio State University, Columbus, UAS(俄亥俄州立大学电气与计算机工程系);Tippie College of Business, The University of Iowa, Iowa City, USA(爱荷华大学蒂皮学院)
Human-Centric Aware UAV Trajectory Planning in Search and Rescue Missions Employing Multi-Objective Reinforcement Learning with AHP and Similarity-Based Experience Replay
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学安全、可靠性与信任跨学科中心);University of Luxembourg(卢森堡大学)
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models
National University of Singapore(新加坡国立大学);Xiamen University(厦门大学);University of Manchester(曼彻斯特大学);Yunnan University(云南大学);Fudan University(复旦大学);Beihang University(北京航空航天大学);China Telecom Artificial Intelligence Technology(Beijing) Co., Ltd.(中国电信人工智能技术(北京)有限公司)