arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2604.16966cs.CRcs.AI

视觉 inception:通过多模态记忆污染在代理推荐系统中妥协长期规划

Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoning

City University of Hong Kong(香港城市大学)

Jiachen Qian

AI总结:

本文提出视觉 inception 攻击,通过污染用户上传的图片在代理推荐系统中影响长期规划,提出 CognitiveGuard 防御框架以降低攻击风险。

摘要:

从静态排序模型到代理推荐系统(Agentic RecSys)的演变使AI代理能够维护长期用户档案并自主规划服务任务。尽管这种范式转变增强了个性化,但引入了对长期记忆(LTM)的依赖性。本文揭示了一种称为“视觉 inception”的威胁。与传统的对抗性攻击不同,视觉 inception 将触发器注入用户上传的图片(如生活方式照片)中,这些触发器作为“睡眠代理”存在于系统内存中。当在未来的规划中被检索时,这些被污染的记忆会劫持代理的推理链,使其朝着对手定义的目标(如推广高利润产品)发展,而无需提示注入。为缓解这一问题,我们提出了CognitiveGuard,一种受人类认知启发的双过程防御框架。它包括一个系统1感知净化器(基于扩散的净化)以净化感官输入,以及一个系统2推理验证器(反事实一致性检查)以检测由记忆驱动的规划中的异常。在模拟的电子商务代理环境中进行的大量实验表明,视觉 inception 实现了约85%的目标命中率(GHR),而CognitiveGuard将这一风险降低到约10%,具有可配置的延迟权衡(轻量模式约1.5秒,完整顺序验证约6.5秒),在我们的设置下不会降低质量。

英文摘要:

The evolution from static ranking models to Agentic Recommender Systems (Agentic RecSys) empowers AI agents to maintain long-term user profiles and autonomously plan service tasks. While this paradigm shift enhances personalization, it introduces a vulnerability: reliance on Long-term Memory (LTM). In this paper, we uncover a threat termed "Visual Inception." Unlike traditional adversarial attacks that seek immediate misclassification, Visual Inception injects triggers into user-uploaded images (e.g., lifestyle photos) that act as "sleeper agents" within the system's memory. When retrieved during future planning, these poisoned memories hijack the agent's reasoning chain, steering it toward adversary-defined goals (e.g., promoting high-margin products) without prompt injection. To mitigate this, we propose CognitiveGuard, a dual-process defense framework inspired by human cognition. It consists of a System 1 Perceptual Sanitizer (diffusion-based purification) to cleanse sensory inputs and a System 2 Reasoning Verifier (counterfactual consistency checks) to detect anomalies in memory-driven planning. Extensive experiments on a mock e-commerce agent environment demonstrate that Visual Inception achieves about 85% Goal-Hit Rate (GHR), while CognitiveGuard reduces this risk to around 10% with configurable latency trade-offs (about 1.5s in lite mode to about 6.5s for full sequential verification), without quality degradation under our setup.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑