Training corpuses for vision language models (VLMs) typically lack sufficient amounts of decision-centric data. This renders off-the-shelf VLMs sub-optimal for decision-making tasks such as in-the-wild device control through graphical user interfaces (GUIs). While training with static demonstrations has shown some promise, we show that such methods fall short for controlling real GUIs due to their failure to deal with real-world stochasticity and non-stationarity not captured in static observational data. This paper introduces a novel autonomous RL approach, called DigiRL, for training in-the-wild device control agents through fine-tuning a pre-trained VLM in two stages: offline RL to initialize the model, followed by offline-to-online RL. To do this, we build a scalable and parallelizable Android learning environment equipped with a VLM-based evaluator and develop a simple yet effective RL approach for learning in this domain. Our approach runs advantage-weighted RL with advantage estimators enhanced to account for stochasticity along with an automatic curriculum for deriving maximal learning signal. We demonstrate the effectiveness of DigiRL using the Android-in-the-Wild (AitW) dataset, where our 1.3B VLM trained with RL achieves a 49.5% absolute improvement -- from 17.7 to 67.2% success rate -- over supervised fine-tuning with static human demonstration data. These results significantly surpass not only the prior best agents, including AppAgent with GPT-4V (8.3% success rate) and the 17B CogAgent trained with AitW data (38.5%), but also the prior best autonomous RL approach based on filtered behavior cloning (57.8%), thereby establishing a new state-of-the-art for digital agents for in-the-wild device control.
WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents
University of Electronic Science and Technology of China(电子科技大学);National University of Singapore(新加坡国立大学);De Artificial Intelligence Lab(德人工智能实验室);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Science and Technology of China(中国科学技术大学)
How many images do I need? Understanding how sample size per class affects deep learning model performance metrics for balanced designs in autonomous wildlife monitoring
School of Science and Technology, University of New England(新英格兰大学科学与技术学院);NSW Department of Primary Industries(新南威尔士州初级产业部);School of Environmental and Rural Science, University of New England(新英格兰大学环境与农村科学学院)
Don't Fool Me Twice: Adapting to Adversity in the Wild with Experience-Driven Reasoning
Department of Engineering Design, Indian Institute of Technology, Madras(印度理工学院工程设计系,马德拉斯);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所)