arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.00536cs.CL
收藏

GUI-KV:借助具备时空感知的KV缓存实现高效GUI智能体

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness

Salesforce AI Research(Salesforce AI研究)

Kung-Hsiang Huang, Haoyi Qiu, Yutong Dai, Caiming Xiong, Chien-Sheng Wu

AI总结:

针对GUI智能体处理长序列高分辨率截图时推理效率低的问题,提出无需重训的即插即用KV缓存压缩方法GUI-KV,结合时空感知技术,在降低运算量的同时提升了准确率。

摘要:

基于视觉-语言模型构建的图形用户界面(GUI)智能体已成为自动化人机工作流的一种颇具前景的方案。但这类智能体在处理高分辨率截图长序列、解决长周期任务时也面临效率低下的难题,导致推理速度慢、成本高且受内存限制。尽管键值(KV)缓存能够缓解这一问题,但在图像占比高的上下文场景中,存储完整缓存的开销令人难以承受。现有的缓存压缩方法效果欠佳,因为它们没有考虑GUI的空间与时间冗余特性。\n在本研究中,我们首先分析了GUI智能体工作负载中的注意力模式,发现与自然图像场景不同,GUI场景下所有Transformer层的注意力稀疏度都普遍偏高。这一发现启发我们提出了一种简单的统一预算分配策略,实验证明该策略比更复杂的随层变化分配方案表现更优。\n在此基础上,我们提出了GUI-KV——一种无需重新训练、可即插即用的GUI智能体KV缓存压缩方法。GUI-KV结合了两项创新技术:(1)空间显著性引导,利用隐藏状态的L2范数增强注意力得分,以更好地保留语义重要的视觉token;(2)时间冗余评分,将先前帧的键投影到当前帧的键子空间中,以优先剪枝冗余历史信息。\n在标准GUI智能体基准测试与多个模型上的实验结果显示,GUI-KV的表现优于现有主流KV压缩基线方法,在适中的缓存预算下,其精度与全缓存精度十分接近。值得注意的是,在AgentNetBench基准的5张截图设置下,GUI-KV相比全缓存基线减少了38.9%的解码浮点运算量(FLOPs),同时将步骤准确率提升了4.1%。这些结果表明,利用GUI特有的冗余特性能够实现高效且可靠的智能体性能。

英文摘要:

Graphical user interface (GUI) agents built on vision-language models have emerged as a promising approach to automate human-computer workflows. However, they also face the inefficiency challenge as they process long sequences of high-resolution screenshots and solving long-horizon tasks, making inference slow, costly and memory-bound. While key-value (KV) caching can mitigate this, storing the full cache is prohibitive for image-heavy contexts. Existing cache-compression methods are sub-optimal as they do not account for the spatial and temporal redundancy of GUIs. In this work, we first analyze attention patterns in GUI agent workloads and find that, unlike in natural images, attention sparsity is uniformly high across all transformer layers. This insight motivates a simple uniform budget allocation strategy, which we show empirically outperforms more complex layer-varying schemes. Building on this, we introduce GUI-KV, a plug-and-play KV cache compression method for GUI agents that requires no retraining. GUI-KV combines two novel techniques: (i) spatial saliency guidance, which augments attention scores with the L2 norm of hidden states to better preserve semantically important visual tokens, and (ii) temporal redundancy scoring, which projects previous frames' keys onto the current frame's key subspace to preferentially prune redundant history. Across standard GUI agent benchmarks and models, GUI-KV outperforms competitive KV compression baselines, closely matching full-cache accuracy at modest budgets. Notably, in a 5-screenshot setting on the AgentNetBench benchmark, GUI-KV reduces decoding FLOPs by 38.9% while increasing step accuracy by 4.1% over the full-cache baseline. These results demonstrate that exploiting GUI-specific redundancies enables efficient and reliable agent performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑