思维链推理有助于移动GUI智能体吗？一项实证研究
Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study
AI总结：
该研究首次实证探究推理增强型视觉语言模型对移动GUI智能体的作用，通过多基准多模型测试发现推理效果存在场景差异，并据此提出了优化移动GUI智能体的相关方向。
摘要：
推理能力已显著提升了视觉语言模型（VLM）在数学解题、编程和视觉问答等领域的性能，但它们对现实世界应用的影响仍不明确。本文首次针对具备推理能力的VLM在移动GUI智能体中的有效性开展实证研究，该领域需要解读复杂屏幕布局、理解用户指令并执行多轮交互。我们评估了两对商用模型——Gemini 2.0 Flash与Claude 3.7 Sonnet，在两个静态基准（ScreenSpot和AndroidControl）及一个交互环境（AndroidWorld）中对比其基础版本与推理增强版本的表现。我们意外发现，Claude 3.7 Sonnet推理模型在AndroidWorld上取得了当前最优性能。但总体而言，推理VLM在静态基准上相比非推理模型仅带来边际提升，在部分智能体设置中甚至会降低性能。值得注意的是，推理与非推理VLM在不同的任务集上失败，说明推理确实存在影响，但其利弊相互抵消。我们将这些不一致性归因于基准和VLM的局限性。基于研究结果，我们从基准、VLM以及动态调用推理VLM的适配性方面，为进一步优化移动GUI智能体提供了洞见。实验数据已公开于https://github.com/LlamaTouch/VLM-Reasoning-Traces。
英文摘要：
Reasoning capabilities have significantly improved the performance of vision-language models (VLMs) in domains such as mathematical problem-solving, coding, and visual question-answering. However, their impact on real-world applications remains unclear. This paper presents the first empirical study on the effectiveness of reasoning-enabled VLMs in mobile GUI agents, a domain that requires interpreting complex screen layouts, understanding user instructions, and executing multi-turn interactions. We evaluate two pairs of commercial models--Gemini 2.0 Flash and Claude 3.7 Sonnet--comparing their base and reasoning-enhanced versions across two static benchmarks (ScreenSpot and AndroidControl) and one interactive environment (AndroidWorld). We surprisingly find the Claude 3.7 Sonnet reasoning model achieves state-of-the-art performance on AndroidWorld. However, reasoning VLMs generally offer marginal improvements over non-reasoning models on static benchmarks and even degrade performance in some agent setups. Notably, reasoning and non-reasoning VLMs fail on different sets of tasks, suggesting that reasoning does have an impact, but its benefits and drawbacks counterbalance each other. We attribute these inconsistencies to the limitations of benchmarks and VLMs. Based on the findings, we provide insights for further enhancing mobile GUI agents in terms of benchmarks, VLMs, and their adaptability in dynamically invoking reasoning VLMs. The experimental data are publicly available at https://github.com/LlamaTouch/VLM-Reasoning-Traces.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
