Vision-language models (VLMs) are increasingly deployed in real-world and embodied settings where safety decisions depend on visual context. However, it remains unclear which visual evidence drives these judgments. We study whether multimodal safety behavior in VLMs can be steered by simple semantic cues. We introduce a semantic steering framework that applies controlled textual, visual, and cognitive interventions without changing the underlying scene content. To evaluate these effects, we propose SAVeS, a benchmark for situational safety under semantic cues, together with an evaluation protocol that separates behavioral refusal, grounded safety reasoning, and false refusals. Experiments across multiple VLMs and an additional state-of-the-art benchmark show that safety decisions are highly sensitive to semantic cues, indicating reliance on learned visual-linguistic associations rather than grounded visual understanding. We further demonstrate that automated steering pipelines can exploit these mechanisms, highlighting a potential vulnerability in multimodal safety systems.
SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Department of Electrical and Computer Engineering, Seoul National University(电子与计算机工程系,首尔国立大学);Daegu Gyeongbuk Institute of Science and Technology(大邱庆州科学技术院);IPAI, AIIS, ASRI, INMC, and ISRC, Seoul National University(IPAI、AIIS、ASRI、INMC 和 ISRC,首尔国立大学)
Explaining Intrinsic Moral Self-Correction with Mechanistic Interpretability
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan(通讯工程研究院,国立台湾大学);MediaTek Research, Taipei, Taiwan(联发科研究,台北,台湾);Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan(电气工程系,国立台湾大学);Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan(电气工程系,国立清华大学);AI Research Center (AINTU), National Taiwan University, Taipei, Taiwan(人工智能研究中心(AINTU),国立台湾大学)
Tracing Moral Foundations in Large Language Models
Department of Computer Science, University of Southern California(南加州大学计算机科学系);Department of Psychology, University of Southern California(南加州大学心理学系);Center for Computational Language Sciences, University of Southern California(南加州大学计算语言科学中心)
Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models
Human Centric ART, University of Rome Tor Vergata(人类中心ART,罗马大学托尔维加塔分校);ILCC, School of Informatics, University of Edinburgh(ILCC,信息学院,爱丁堡大学);School of Computer Science, University of Sheffield(计算机科学学院,谢菲尔德大学);Almawave S.p.A.(Almawave公司)
ACE-$M^3$: Automatic Capability Evaluator for Multimodal Medical Models
East China Normal University(华东师范大学);Hasso Plattner Institute/University of Potsdam(哈索·普拉特纳研究所/波茨坦大学);East China University of Science and Technology(华东理工大学)
Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attacks
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Ant Group(蚂蚁集团)