明智行动：在代理多模态模型中培养元认知工具使用
Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models
AI总结：
本文提出HDPO框架，通过解耦优化通道提升代理多模态模型的元认知工具使用能力，减少工具调用并提升推理准确性。
摘要：
代理多模态模型的出现使系统能够主动与外部环境交互。然而，当前代理存在深刻的元认知缺陷：难以在利用内部知识和查询外部工具之间做出决策。因此，它们经常陷入盲目工具调用，即使查询可通过原始视觉上下文解决。这种病理行为导致严重的延迟瓶颈，并注入额外噪声，干扰合理推理。现有的强化学习协议试图通过标量奖励来缓解这一问题，但这种联合形式导致不可调和的优化困境：强烈的惩罚抑制了必要的工具使用，而温和的惩罚则被准确性奖励的方差所吞噬，使其无法对抗工具过度使用。为超越这一瓶颈，我们提出HDPO，将工具效率从竞争性标量目标重新定义为严格条件性目标。通过摒弃奖励标量化，HDPO维护两个正交的优化通道：一个准确性通道，最大化任务正确性；一个效率通道，通过条件优势估计在准确轨迹内强制执行执行经济性。这种解耦架构自然诱导一个认知课程，迫使代理首先掌握任务解决，然后再优化自我依赖性。广泛的评估显示，所得到的模型Metis在减少工具调用数量的同时，同时提升了推理准确性。
英文摘要：
The advent of agentic multimodal models has empowered systems to actively interact with external environments. However, current agents suffer from a profound meta-cognitive deficit: they struggle to arbitrate between leveraging internal knowledge and querying external utilities. Consequently, they frequently fall prey to blind tool invocation, resorting to reflexive tool execution even when queries are resolvable from the raw visual context. This pathological behavior precipitates severe latency bottlenecks and injects extraneous noise that derails sound reasoning. Existing reinforcement learning protocols attempt to mitigate this via a scalarized reward that penalizes tool usage. Yet, this coupled formulation creates an irreconcilable optimization dilemma: an aggressive penalty suppresses essential tool use, whereas a mild penalty is entirely subsumed by the variance of the accuracy reward during advantage normalization, rendering it impotent against tool overuse. To transcend this bottleneck, we propose HDPO, a framework that reframes tool efficiency from a competing scalar objective to a strictly conditional one. By eschewing reward scalarization, HDPO maintains two orthogonal optimization channels: an accuracy channel that maximizes task correctness, and an efficiency channel that enforces execution economy exclusively within accurate trajectories via conditional advantage estimation. This decoupled architecture naturally induces a cognitive curriculum-compelling the agent to first master task resolution before refining its self-reliance. Extensive evaluations demonstrate that our resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously elevating reasoning accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
