ICA: 信息感知的信用分配用于基于视觉的长周期信息寻求智能体
ICA: Information-Aware Credit Assignment for Visually Grounded Long-Horizon Information-Seeking Agents
AI总结：
本文提出ICA方法，通过视觉快照和信息层面信用分配，提升开放网络环境中信息寻求智能体的性能。
摘要：
尽管强化学习训练的信息寻求智能体表现出强大的性能，但在开放的网络环境中学习仍然严重受限于低信噪比反馈。基于文本的解析器经常丢弃布局语义并引入无结构噪声，而长周期训练通常依赖于稀疏结果奖励，这些奖励会掩盖哪些检索操作实际上重要。我们提出了一种视觉原生的搜索框架，将网页表示为视觉快照，使智能体能够利用布局提示快速定位显著证据并抑制干扰项。为了有效学习这些高维观察，我们引入了信息感知的信用分配（ICA），一种事后方法，通过后验分析估计每个检索快照对最终结果的贡献，并将密集学习信号回传到关键搜索阶段。与基于GRPO的训练流程集成，我们的方法在多样化的信息寻求基准上持续优于基于文本的基线，提供了证据表明，使用视觉快照接地和信息层面的信用分配可以缓解开放网络环境中的信用分配瓶颈。代码和数据集将在https://github.com/pc-inno/ICA_MM_deepsearch.git发布。
英文摘要：
Long-horizon reinforcement learning for information seeking agents remains difficult because terminal rewards reveal whether the final answer is correct, but not which acquired information enabled it. This difficulty is amplified by text-derived webpage observations, where parsing, truncation, and summarization often produce incomplete and unstable content representations across trajectories. We propose an evidence-centric framework for web agent learning that represents information acquired through tools as identifiable units for comparison across trajectories. In particular, fetched webpages are represented as rendered snapshots, preserving layout and multimodal content as stable content-level observations. Building on these units, we introduce Information-Aware Credit Assignment, a post hoc reward propagation method that estimates turn-level utility scores from rollout success rates and assigns dense rewards to intermediate steps that introduced high-utility information. Integrated with GSPO, our method consistently improves performance on BrowseComp, GAIA, Xbench-DS, and Seal-0. Code and datasets will be released at https://github.com/pc-inno/ICA_MM_deepsearch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
