MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion
Institute of Parallel and Distributed Systems (IPADS), Shanghai Jiao Tong University（上海交通大学并行与分布式系统研究所（IPADS））；Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；National Innovation Institute of High-end Smart Appliances（国家高端智能家电创新研究院）
Mobile agents can autonomously complete user-assigned tasks through GUI interactions. However, existing mainstream evaluation benchmarks, such as AndroidWorld, operate by connecting to a system-level Android emulator and provide evaluation signals based on the state of system resources. In real-world mobile-agent scenarios, however, many third-party applications do not expose system-level APIs to determine whether a task has succeeded, leading to a mismatch between benchmarks and real-world usage and making it difficult to evaluate model performance accurately. To address these issues, we propose MobiFlow, an evaluation framework built on tasks drawn from arbitrary third-party applications. Using an efficient graph-construction algorithm based on multi-trajectory fusion, MobiFlow can effectively compress the state space, support dynamic interaction, and better align with real-world third-party application scenarios. MobiFlow covers 20 widely used third-party applications and comprises 240 diverse real-world tasks, with enriched evaluation metrics. Compared with AndroidWorld, MobiFlow's evaluation results show higher alignment with human assessments and can guide the training of future GUI-based models under real workloads.
HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring
School of Computing and Information Systems, University of Melbourne, Australia（墨尔本大学计算机与信息系统学院）；School of Computer Science, University of Auckland, New Zealand（奥克兰大学计算机科学学院）
GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
Yale University（耶鲁大学）；Zhejiang University（浙江大学）；China University of Geosciences（中国地质大学）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Tongji University（同济大学）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Time to Embrace Natural Language Processing (NLP)-based Digital Pathology: Benchmarking NLP- and Convolutional Neural Network-based Deep Learning Pipelines
University of Science and Technology of China（中国科学技术大学）；UNSW Sydney（新南威尔士大学）；Genmab Inc.（Genmab公司）
NavTrust: Benchmarking Trustworthiness for Embodied Navigation
Trustworthy Autonomous Systems Laboratory at the University of California, Riverside（加州大学河滨分校可信自主系统实验室）；University of Michigan（密歇根大学）；Workday；University of Southern California（南加州大学）；Texas A&M University（德克萨斯A&M大学）；Lehigh University（莱斯大学）
Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents
Beijing University of Posts and Telecommunications（北京邮电大学）；State Key Laboratory for General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，字节跳动公司人工智能研究院）；China University of Geosciences (Beijing)（中国地质大学（北京））；Beijing Institute of Technology（北京理工大学）
Benchmarking LLM Judges for Mobile Agent Evaluation
Mila – Québec AI Institute（米拉-魁北克人工智能研究所）；Concordia University（康考迪亚大学）；University of Toronto（多伦多大学）；Shanghai University（上海大学）；McMaster University（麦克马斯特大学）
Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset
Norwegian Institute for Bioeconomy Research (NIBIO)（挪威生物经济研究所）；University of Cambridge（剑桥大学）；Jan Evangelista Purkyně University in Ústí n. L.（拉贝河畔乌斯季扬·埃万杰利斯塔·普尔基涅大学）；University of Freiburg（弗莱堡大学）；Georg-August-Universität Göttingen（哥廷根大学）；TU Wien（维也纳工业大学）；Swiss Federal Institute for Forest, Snow and Landscape Research WSL（瑞士联邦森林、雪与景观研究所）；University of British Columbia（不列颠哥伦比亚大学）；Ghent University（根特大学）；Heidelberg University（海德堡大学）；University of Eastern Finland（东芬兰大学）；Silva Tarouca Research Institute（席尔瓦·塔劳卡研究所）；Forest Research Institute（森林研究所）；Commonwealth Scientific and Industrial Research Organization (CSIRO)（联邦科学与工业研究组织）；Charles Darwin University（查尔斯·达尔文大学）；University College of London（伦敦大学学院）；Natural Resources Institute Finland (Luke)（芬兰自然资源研究所）；Council for Agricultural Research and Economics（农业研究与经济委员会）；University of Lethbridge（莱斯布里奇大学）；National Research Council – Institute of BioEconomy（国家研究委员会生物经济研究所）；Free University of Bolzano（博尔扎诺自由大学）
MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments
Department of Computer Science & Technology, Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint Center for ML（计算机科学与技术系、人工智能研究院、BNRist中心、THBI实验室、清华-博世联合机器学习中心）；Shanghai Key Lab. of Multidimensional Info. Processing（上海多维信息处理重点实验室）
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
XiaoMi AI Lab（小米AI实验室）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；University of Electronic Science and Technology of China（电子科技大学）
WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networks
College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港理工大学电子与计算机工程系）
