Xiaomi-GUI-0 技术报告
Xiaomi-GUI-0 Technical Report
AI总结：
提出Xiaomi-GUI-0，一种在真实移动设备上训练和评估的原生多模态GUI智能体，通过真实设备主导的混合基础设施、多源训练数据和渐进式三阶段训练，在真实任务中实现72.0%成功率，显著提升执行稳定性和异常状态识别。
AI中文摘要：
图形用户界面（GUI）智能体基于视觉语言模型，通过点击、滑动、文本输入和导航等界面操作，在真实应用中端到端完成用户任务。然而，现有GUI智能体主要在离线轨迹、模拟环境和标准化基准上训练和评估，这些与真实应用在界面布局、交互逻辑和异常状态分布上存在显著差异，无法真实反映实际使用中的执行稳定性——账户状态、权限对话框、支付认证和风险控制不断重塑状态分布，导致基准分数与实际可用性之间存在持续差距。为弥合这一差距，我们提出Xiaomi-GUI-0，一种面向真实移动环境的原生多模态GUI智能体，在真实设备闭环中训练和评估。其核心是真实设备主导的混合基础设施，物理设备作为主要执行环境，沙箱提供辅助支持，使得数据收集、训练、推理和评估共享接近真实部署的执行分布。我们构建了多源训练数据，涵盖高频头部任务、长尾意图的高泛化数据以及用于反思和记忆的能力增强数据，并引入错误驱动数据飞轮，将失败轨迹转化为修正动作、反思解释和恢复演示。模型通过监督微调、步骤级强化学习和智能体强化学习的三阶段渐进式流水线进行训练。在公开基准和内部RealMobile上的评估显示，Xiaomi-GUI-0在RealMobile上达到72.0%的成功率，在AndroidWorld上达到78.9%，同时在真实任务中大幅提升了执行稳定性和异常状态识别能力。
英文摘要：
Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation. However, existing GUI agents are trained and evaluated largely on offline trajectories, simulated environments, and standardized benchmarks. These differ substantially from real applications in interface layout, interaction logic, and abnormal-state distribution, and cannot faithfully characterize execution stability in real-world use, where account states, permission dialogs, payment authentication, and risk control continually reshape the state distribution and open a persistent gap between benchmark scores and real usability. To close this gap, we propose Xiaomi-GUI-0, a native multimodal GUI agent for real mobile environments, trained and evaluated within a real-device closed loop. At its core is a real-device-dominant hybrid infrastructure, where physical devices are the primary execution environment and sandboxes provide auxiliary support, so that data collection, training, rollout, and evaluation share an execution distribution close to real deployment. We construct multi-source training data spanning high-frequency head tasks, high-generalization data for long-tail intents, and capability-enhancement data for reflection and memory, and introduce an error-driven data flywheel that turns failure trajectories into corrected actions, reflective explanations, and recovery demonstrations. The model is trained through a progressive three-stage pipeline of supervised fine-tuning, step-level reinforcement learning, and agentic reinforcement learning. Evaluated on public benchmarks and our in-house RealMobile, Xiaomi-GUI-0 achieves 72.0% success on RealMobile and 78.9% on AndroidWorld, while substantially improving execution stability and abnormal-state recognition in real-world tasks.
