PriMobiBench：表征VLM驱动的移动GUI代理中的视觉隐私泄露
PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agents
浏览论文内容
中文总结 AI 辅助
针对VLM驱动的移动GUI代理的视觉隐私泄露问题，提出首个基准PriMobiBench和数据集MobiLeak，量化了高成功率的敏感信息提取与用户画像风险，并验证了掩码缓解方案的有效性。
中文摘要 AI 辅助
移动GUI代理越来越依赖视觉语言模型（VLM）通过解释屏幕截图流来自动化智能手机任务。然而，这种设计引入了严重且未被充分探索的隐私风险，包括敏感屏幕信息的直接泄露和意外的用户画像构建。缺乏标准化基准使得在现实移动代理工作流中量化这些风险变得困难。为解决这一空白，我们提出了PriMobiBench，这是首个系统评估截图驱动移动代理中隐私泄露和视觉画像的基准。它提供了数据生成、代理轨迹构建和多模型评估的统一流程。我们还引入了MobiLeak，一个包含来自16个应用的执行轨迹的数据集，覆盖25个隐私属性，包含2,960个嵌入的隐私实例。我们的结果揭示了重大风险：（1）VLM可以直接提取敏感信息，成功率高达82.5%；（2）除了显式泄露外，它们可以从聚合的视觉证据中推断用户画像，成功率约为70%。我们进一步提出了一种缓解措施，在云端处理前对隐私敏感但与任务无关的UI元素进行掩码，将画像成功率降低高达58%，同时仅造成约8%的性能损失。总体而言，我们的工作为移动GUI代理中的视觉隐私风险提供了首个系统基准，证明了泄露和画像在高度令人担忧的水平上是可行的，并提供了实用的缓解方向。
英文摘要
Mobile GUI agents increasingly rely on Vision-Language Models (VLMs) to automate smartphone tasks by interpreting screenshot streams. However, this design introduces serious and underexplored privacy risks, including direct leakage of sensitive on-screen information and unintended user profiling. The absence of standardized benchmarks makes it difficult to quantify these risks in realistic mobile agent workflows. To address this gap, we propose PriMobiBench, the first benchmark for systematically evaluating privacy leakage and visual profiling in screenshot-driven mobile agents. It provides a unified pipeline for data generation, agent trajectory construction, and multi-model evaluation. We also introduce MobiLeak, a dataset of execution traces from 16 apps, covering 25 privacy attributes with 2,960 embedded privacy instances. Our results reveal substantial risks: (1) VLMs can directly extract sensitive information with up to 82.5% success rate; (2) beyond explicit leakage, they can infer user profiles from aggregated visual evidence with approximately 70% success. We further propose a mitigation that masks privacy-sensitive but task-irrelevant UI elements before cloud processing, reducing profiling success by up to 58% with only approximately 8% performance loss. Overall, our work provides the first systematic benchmark for visual privacy risks in mobile GUI agents, demonstrates that both leakage and profiling are feasible at a highly concerning level, and offers a practical direction for mitigation.
发表机构
- Tsinghua University（清华大学）
- Shandong University（山东大学）
- Wuhan University（武汉大学）
机构由 AI 辅助整理，请以论文原文为准。
