arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.13372cs.CR
收藏

面向安全的世界模型:风险、防御与评估

Towards Safe World Models: Risks, Defenses, and Evaluation

Beijing University of Posts and Telecommunications(北京邮电大学);Beihang University(北京航空航天大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校)

Xiaoma Wang, Litian Zhang, Xinze Shi, Zhengxuan Hu, Zihan Yang, Mingqi Ren, You Wu, Bingyu Yan, Zelong Yu, Zhenjie Su, Yuming Shang, Xi Zhang, Chaozhuo Li, Philip S. Yu

AI总结:

本研究梳理世界模型安全相关文献,构建六阶段生命周期框架,分析其风险、防御与评估,明确当前研究的支撑情况与待解决问题,为安全世界模型的构建提供方向。

摘要:

世界模型通过预测动作条件下的结果,支持规划、控制、基于想象经验的学习以及模型介导的评估。其安全性无法仅从预测精度推断:观测、状态与记忆、动力学、评估、执行或反馈中的错误会传播至不安全决策,并在更新中持续存在。本研究围绕涵盖数据与观测、状态与记忆、动力学与想象、评估与规划、动作与控制、反馈与更新的六阶段生命周期,梳理世界模型安全相关文献,综合非对抗性故障、攻击、防御、评估协议、基准及部署条件,区分证据直接性与结果层级。当前证据 corpus 包含 163 篇引用文献,本研究运用该框架识别主张的支撑情况、归因未明确之处,以及连接模型级行为与闭环及操作安全所需的测量指标。

英文摘要:

World models support planning, control, learning from imagined experience, and model-mediated evaluation by predicting action-conditioned consequences. Their safety cannot be inferred from predictive accuracy alone: errors in observations, state and memory, dynamics, evaluation, execution, or feedback can propagate into unsafe decisions and persist through updates. This work organizes the literature on world-model safety around a six-stage lifecycle covering data and observations, state and memory, dynamics and imagination, evaluation and planning, action and control, and feedback and update. We synthesize non-adversarial failures, attacks, defenses, evaluation protocols, benchmarks, and deployment conditions, and distinguish evidence directness from outcome level. The current evidence corpus contains 163 cited works. We use the framework to identify where claims are supported, where attribution remains unresolved, and which measurements are needed to connect model-level behavior to closed-loop and operational safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑