arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.34384cs.LGcs.AIcs.CLcs.CV
收藏

AndroidFlux:评估移动使用多模态智能体的故障恢复能力与奖励建模

AndroidFlux: Evaluating and Reward Modeling Failure Recovery Capabilities in Mobile Use Multimodal Agents

Georgia Institute of Technology(佐治亚理工学院);Konkuk University(建国大学)

Gyubeum Lim, Yemo Koo, Ghunwoo Jang, Zsolt Kira

AI总结:

本研究推出基于AndroidWorld的故障恢复基准AndroidFlux,配套奖励建模基准与离线RL数据集,发现移动多模态智能体恢复性能与干净启动成功率部分相关,显式推理仅对GUI领域特定RL训练的模型有帮助。

摘要:

移动使用多模态智能体通常通过从干净初始状态的任务成功度进行评估,这无法单独衡量其从中间错误中恢复的能力。由于移动环境动态变化,通用移动使用智能体必须在意外状态和未见环境中仍能稳健完成任务,这需要明确智能体何时何地出错,以及能否从错误及后续状态中恢复。为评估该能力,我们推出基于AndroidWorld构建的故障恢复基准AndroidFlux,它将记录的轨迹恢复到首次可恢复错误之前、之时及之后的受控检查点,使不同智能体从相同设备状态和历史继续执行。在这些检查点上,配套的奖励建模基准利用下游rollout结果评估恢复动作偏好。我们还发布了一个离线强化学习(RL)数据集,可同时提升错误恢复能力和整体任务成功率。评估显示,恢复性能仅与干净启动成功率部分相关,且显式推理仅对经GUI领域特定RL训练的模型有帮助。

英文摘要:

Mobile-use multimodal agents are typically evaluated by task success from a clean initial state, which does not isolate their ability to recover from intermediate errors. Because mobile environments change dynamically, a general mobile-use agent must complete tasks robustly even from unexpected states and in unseen environments. This requires knowing when and where an agent makes errors, and whether it can recover from errors and subsequent states. To evaluate this capability, we introduce AndroidFlux, a failure-recovery benchmark built on AndroidWorld. AndroidFlux restores recorded trajectories to controlled checkpoints before, at, and after the first recoverable error, so that different agents continue from the same device state and history. On these checkpoints, a complementary reward-modeling benchmark evaluates recovery-action preferences using downstream rollout outcomes. We further release an offline reinforcement learning (RL) dataset that improves both error recovery and overall task success. Our evaluations show that recovery performance only partially correlates with clean-start success, and that explicit reasoning helps only models trained with GUI-domain specific RL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑