arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.18697cs.LGcs.CRcs.RO
收藏

通过数据投毒进行隐蔽的世界模型操纵

Stealthy World Model Manipulation via Data Poisoning

Department of Computer Science(计算机科学系)

Yibin Hu, Xiaolin Sun, Zizhan Zheng

AI总结:

提出SWAAP框架,通过两阶段数据投毒(双层级优化寻找有害目标模型+梯度匹配隐蔽实现)操纵学习到的世界模型,导致规划性能显著下降,且能规避多种防御检测。

摘要:

基于模型的学习智能体使用学习到的世界模型来预测未来状态、规划行动并适应新环境。然而,从收集的经验中更新世界模型的过程创造了一个训练时攻击面:对抗性投毒的微调轨迹可以操纵学习到的动力学,从而破坏下游规划。在本文中,我们提出了SWAAP,这是第一个针对学习到的世界模型的两阶段数据投毒框架。在第一阶段,SWAAP利用过渡梯度定理实现的一阶双层优化,识别出一个有害的目标世界模型,该模型在规划下诱导低回报行为,同时保持接近干净动力学。在第二阶段,SWAAP通过隐蔽约束的梯度匹配实现该目标,仅修改有限比例的微调过渡目标,使得诱导的训练梯度将受害者模型引向对抗目标,同时预测误差正则化器鼓励投毒目标保持接近世界模型的自然近似误差。为了评估攻击的隐蔽性,我们在投毒管道的三个阶段评估了防御和可检测性:投毒过渡的预训练检测、微调期间的鲁棒训练以及测试时对结果世界模型的监控。在多种连续控制任务中,SWAAP导致显著的性能下降,同时保持投毒过渡接近干净数据,并规避了评估的非自适应残差/CUSUM/TRIM风格防御。这些结果揭示了世界模型适应管道中的实际漏洞,并强调了需要保护世界模型训练数据和所学动力学的鲁棒性方法。

英文摘要:

Model-based learning agents use learned world models to predict future states, plan actions, and adapt to new environments. However, the process of updating world models from collected experience creates a training-time attack surface: adversarially poisoned fine-tuning trajectories can manipulate the learned dynamics and thereby corrupt downstream planning. In this paper, we propose SWAAP, the first two-stage data poisoning framework for learned world models. In the first stage, SWAAP identifies a harmful target world model that induces low-return behavior under planning while remaining close to clean dynamics, using first-order bilevel optimization enabled by a transition-gradient theorem. In the second stage, SWAAP realizes this target through stealth-constrained gradient matching, modifying only a limited fraction of fine-tuning transition targets so that the induced training gradients steer the victim model toward the adversarial target, while a prediction-error regularizer encourages the poisoned targets to remain close to the world model's natural approximation error. To assess attack stealthiness, we evaluate defenses and detectability across three stages of the poisoning pipeline: pre-training detection of poisoned transitions, robust training during fine-tuning, and test-time monitoring of the resulting world model. Across diverse continuous-control tasks, SWAAP causes substantial performance degradation while keeping poisoned transitions close to clean data and evading the evaluated non-adaptive residual/CUSUM/TRIM-style defenses. These results reveal a practical vulnerability in world-model adaptation pipelines and highlight the need for robustness methods that protect both world-model training data and learned dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑