世界模型通过预测动作条件下的结果,支持规划、控制、基于想象经验的学习以及模型介导的评估。其安全性无法仅从预测精度推断:观测、状态与记忆、动力学、评估、执行或反馈中的错误会传播至不安全决策,并在更新中持续存在。本研究围绕涵盖数据与观测、状态与记忆、动力学与想象、评估与规划、动作与控制、反馈与更新的六阶段生命周期,梳理世界模型安全相关文献,综合非对抗性故障、攻击、防御、评估协议、基准及部署条件,区分证据直接性与结果层级。当前证据 corpus 包含 163 篇引用文献,本研究运用该框架识别主张的支撑情况、归因未明确之处,以及连接模型级行为与闭环及操作安全所需的测量指标。
英文摘要:
World models support planning, control, learning from imagined experience, and model-mediated evaluation by predicting action-conditioned consequences. Their safety cannot be inferred from predictive accuracy alone: errors in observations, state and memory, dynamics, evaluation, execution, or feedback can propagate into unsafe decisions and persist through updates. This work organizes the literature on world-model safety around a six-stage lifecycle covering data and observations, state and memory, dynamics and imagination, evaluation and planning, action and control, and feedback and update. We synthesize non-adversarial failures, attacks, defenses, evaluation protocols, benchmarks, and deployment conditions, and distinguish evidence directness from outcome level. The current evidence corpus contains 163 cited works. We use the framework to identify where claims are supported, where attribution remains unresolved, and which measurements are needed to connect model-level behavior to closed-loop and operational safety.
TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations
Institute of Artificial Intelligence (TeleAI) of China Telecom(中国电信人工智能研究院(TeleAI));Shanghai Jiao Tong University(上海交通大学);Sichuan University(四川大学);University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院);University of Science and Technology of China(中国科学技术大学);Peking University(北京大学);Harbin Institute of Technology(哈尔滨工业大学)
SoK: The Security-Safety Continuum of Multimodal Foundation Models through Information Flow and Global Game-Theoretic Analysis of Asymmetric Threats
CSIRO’s Data61(CSIRO数据61);University of New South Wales(新南威尔士大学);Johns Hopkins University(约翰斯·霍普金斯大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Adelaide University(阿德莱德大学);Responsible AI Research (RAIR) Centre, Adelaide University(阿德莱德大学负责任人工智能研究(RAIR)中心)
SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models
Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Harbin Institute of Technology, Shenzhen, Shenzhen, China(深圳视觉目标检测与识别重点实验室,哈尔滨工业大学,哈尔滨工业大学,深圳,深圳,中国)
Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution
City University of Hong Kong(香港城市大学);Beijing University of Posts and Telecommunications(北京邮电大学);Wuhan University(武汉大学);Beihang University(北京航空航天大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning
School of Computing(计算学院);National University of Singapore(新加坡国立大学);School of Computer Science and Engineering(计算机科学与工程学院);Nanyang Technological University(南洋理工大学);Department of Computing(计算系);The Hong Kong Polytechnic University(香港理工大学)
SteerProbe: Learning to Bypass Safety Steering in Vision-Language Models
The Hong Kong Polytechnic University(香港理工大学);Anhui University of Technology(安徽工业大学);Nanyang Technological University(南洋理工大学);Nanjing University of Science and Technology(南京理工大学)
SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳研究院);Southwestern University of Finance and Economics(西南财经大学);Engineering Research Center of Intelligent Finance, Ministry of Education(教育部智能金融工程研究中心);Tongji University(同济大学)
Refusal is Not Safety! Benchmarking Latent Safety Risks of LLM-Driven Content Humorization
Beijing Institute of Technology(北京理工大学);Chengdu Institute of Computer Applications, Chinese Academy of Sciences(中国科学院成都计算机应用研究所);University of Chinese Academy of Sciences(中国科学院大学);Yangtze Delta Region Institute of Tsinghua University, Zhejiang(清华大学长三角研究院(浙江))
When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills
The University of Sydney(悉尼大学);University of Queensland(昆士兰大学);Southeast University(东南大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University(电子工程系,北京大学);School of Mathematical Sciences, Peking University(数学系,北京大学);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(通用人工智能国家重点实验室,智能科学与技术学校,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)
PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
Beijing Institute of AI Safety and Governance (Beijing-AISI)(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Superalignment(北京安全人工智能与超对齐重点实验室);BrainCog Lab, CASIA(CASIA脑认知实验室)