MTA-Zero:通过在线问答求解器协同训练实现自改进的多模态深度搜索
MTA-Zero: Self-Improving Multimodal Deep Search via Online QA-Solver Co-training
AI总结:
MTA-Zero是无需预计算多跳语料库的在线多模态问答求解器协同训练框架,通过GRPO优化生成器与求解器策略,在六个多模态深度搜索基准上显著提升了Qwen系列求解器性能,达到与GPT-5相当的水平。
摘要:
多模态大语言模型(MLLMs)已成为强大的视觉推理器,但在需要图像 grounding、工具选择和网络证据整合的开放世界多步搜索中仍存在局限。现有方法通常构建并过滤固定的离线语料库,其问题分布无法随目标深度搜索智能体的改进而自适应。基于MTA-Agent的图像 grounding 问答构建方法,我们提出MTA-Zero,这是一种无需预计算多跳语料库的、用于自改进多模态深度搜索智能体的在线多模态问答求解器协同训练框架。从图像 grounding 的视觉问答(VQA)种子上下文出发,可训练的问答生成器在线检索证据并构建经过验证的多跳问题;独立的深度搜索求解器则选择工具、收集证据并回答生成的问题。为将生成聚焦于求解器的学习前沿,我们从求解器的分组 rollout 成功率中为问答生成器推导了 Goldilocks 奖励:所有 rollout 都能解决或都不能解决的问题获得零奖励,而接近当前前沿的问题则被优先选择。我们使用GRPO独立优化两个策略,不共享参数或梯度。在六个多模态深度搜索基准上,MTA-Zero将Qwen3-VL-8B求解器的平均准确率从36.91%提升至51.58%,在相同智能体工作流程下达到与GPT-5相当的性能;同一框架还将更新的Qwen3.6-27B求解器提升了10.82个百分点。这些结果表明,经过验证的在线问题生成为持续改进多模态深度搜索智能体提供了特定于策略的课程。
英文摘要:
Multimodal large language models (MLLMs) have become strong visual reasoners, yet remain limited in open-world, multi-step search that requires grounding images, selecting tools, and integrating web evidence. Existing approaches typically construct and filter fixed offline corpora whose question distributions cannot adapt as the target deep-search agent improves. Building on MTA-Agent's image-grounded QA construction, we present MTA-Zero, an online multimodal QA-Solver co-training framework for self-improving multimodal deep-search agents that requires no precomputed multi-hop corpus. Starting from image-grounded VQA seed contexts, a trainable QA Generator retrieves evidence and constructs verified multi-hop questions online. A separate deep-search Solver then selects tools, gathers evidence, and answers the generated questions. To focus generation on the Solver's learning frontier, we derive a Goldilocks reward for the QA Generator from the Solver's group-wise rollout success rate: questions solved by all rollouts or by none receive zero reward, whereas questions near the current frontier are favored. We optimize the two policies independently with GRPO, without sharing parameters or gradients. Across six multimodal deep-search benchmarks, MTA-Zero improves a Qwen3-VL-8B Solver from 36.91% to 51.58% mean accuracy and achieves performance comparable to GPT-5 under the same agent workflow. The same framework also improves a newer Qwen3.6-27B Solver by 10.82 points. These results show that verified online question generation provides a policy-specific curriculum for continually improving multimodal deep-search agents.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。