Exploratory GUI testing is essential for software quality but suffers from high manual costs. While Multi-modal Large Language Model (MLLM) agents excel in navigation, they fail to autonomously discover defects due to two core challenges: \textit{Goal-Oriented Masking}, where agents prioritize task completion over reporting anomalies, and \textit{Execution-Bias Attribution}, where system defects are misidentified as agent errors. To address these, we first introduce \textbf{GUITestBench}, the first interactive benchmark for this task, featuring 143 tasks across 26 defects. We then propose \textbf{GUITester}, a multi-agent framework that decouples navigation from verification via two modules: (i) a \textit{Planning-Execution Module (PEM)} that proactively probes for defects via embedded testing intents, and (ii) a \textit{Hierarchical Reflection Module (HRM)} that resolves attribution ambiguity through interaction history analysis. GUITester achieves an F1-score of 48.90\% (Pass@3) on GUITestBench, outperforming state-of-the-art baselines (33.35\%). Our work demonstrates the feasibility of autonomous exploratory testing and provides a robust foundation for future GUI quality assurance~\footnote{Our code is now available in~\href{https://github.com/ADaM-BJTU/GUITestBench}{https://github.com/ADaM-BJTU/GUITestBench}}.
DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis
School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);School of Software Engineering, Sun Yat-sen University(中山大学软件学院);Lingnan College, Sun Yat-sen University(中山大学岭南学院)
TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains
Shanghai Key Laboratory of Computer Software Testing and Evaluating, Shanghai Development Center of Computer Software Technology(上海软件测试与评估关键实验室、上海计算机软件技术发展中心);Shanghai Normal University(上海师范大学);Shanghai Artificial Intelligence Lab(上海人工智能实验室);Shanghai University(上海大学);Shanghai Jiao Tong University(上海交通大学)
Meta AI Research(Meta人工智能研究院);Carnegie Mellon University(卡内基梅隆大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
像图像-目标导航这样的现实长期任务涉及探索和利用阶段。由于目标的图像,一个具体的代理必须探索发现目标,即,使用学习的先验知识进行有效搜索。一旦目标被发现,智能体必须准确地校准到目标的最后一英里的导航。与任何强大的系统一样,探索性目标发现和利用性最后一英里导航之间的切换可以更好地从错误中恢复。根据这些直观的导轨,我们提出了SLING,以提高现有的图像目标导航系统的性能。作为对现有方法的补充,我们专注于最后一英里导航,并利用神经描述符来利用问题的底层几何结构。通过简单但有效的开关,我们可以轻松地将SLING与启发式、强化学习和神经模块化策略联系起来。在标准化的图像目标导航基准(Hahn et al. 2021)上,我们提高了策略,场景和情节复杂性的性能,将最先进的成功率从45%提高到55%。除了真实感模拟,我们在三个物理场景中进行了真实的机器人实验,并发现这些改进可以很好地转移到真实环境中。