ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；University College London（伦敦大学学院）；AI Lab, The Yangtze River Delta（长江三角洲人工智能实验室）
Anjie Liu, Yan Song, Zhixun Chen, Ziqin Gong, Zhongwei Yu, Jun Wang
工具增强的视觉语言智能体可以通过OCR、检测、分割等工具获取外部感知证据，但执行每个提议的工具调用成本高昂且有时不必要。我们研究了预调用控制问题：在ReAct风格的VLM智能体提出感知工具调用后，是否应执行该调用，还是在其输出进入上下文之前跳过？在五个基准测试中，我们发现基线智能体表现出较差的局部选择性：有益和有害调用的发生率相近（11.8% vs. 9.9%），而大多数调用不会改变即时强制答案预测。我们引入了ToolGate，一个轻量级外部控制器，它根据轨迹文本和简单的结构特征预测执行/跳过决策。在两个Qwen3-VL骨干网络上，ToolGate将令牌成本降低到无限制ReAct基线的64-69%，同时保持跨域设置的平均准确率。在Qwen3-VL-30B上进行匹配域轨迹训练后，它进一步将平均准确率提高了1.65个百分点。这些结果表明，工具增强的VLM智能体不仅受益于更好的感知工具，还受益于对工具输出何时值得付费的显式控制。
英文摘要：
Tool-augmented vision-language agents can acquire external perceptual evidence through OCR, detection, segmentation, and other tools, but executing every proposed tool call is costly and sometimes unnecessary. We study the pre-call control problem: after a ReAct-style VLM agent proposes a perceptual tool call, should the call be executed, or skipped before its output enters the context? Across five benchmarks, we find that the baseline agent exhibits poor local selectivity: helpful and harmful calls occur at similar rates (11.8% vs. 9.9%), while most calls do not change the immediate forced-answer prediction. We introduce ToolGate, a lightweight external controller that predicts execute/skip decisions from trajectory text and simple structural features. Across two Qwen3-VL backbones, ToolGate reduces token cost to 64-69% of the unrestricted ReAct baseline while preserving average accuracy in cross-domain settings. With matched-domain trajectory training on Qwen3-VL-30B, it further improves average accuracy by 1.65 points. These results show that tool-augmented VLM agents benefit not only from better perceptual tools, but also from explicit control over when tool outputs are worth paying for.
Google Research（谷歌研究院）；Carnegie Mellon University（卡内基梅隆大学）
We propose a new paradigm to help Large Language Models (LLMs) generate more accurate factual knowledge without retrieving from an external corpus, called RECITation-augmented gEneration (RECITE). Different from retrieval-augmented language models that retrieve relevant documents before generating the outputs, given an input, RECITE first recites one or several relevant passages from LLMs' own memory via sampling, and then produces the final answers. We show that RECITE is a powerful paradigm for knowledge-intensive NLP tasks. Specifically, we show that by utilizing recitation as the intermedi…
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Alibaba Group（阿里巴巴集团）；Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；Shenzhen Intelligent Strong Technology Co., Ltd（深圳智能强科技有限公司）
LightMem: Lightweight and Efficient Memory-Augmented Generation
Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；State Key Lab. for Novel Software Technology, Nanjing University, P.R. China（南京大学软件新技术国家重点实验室，中国）
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Exploration in environments with sparse feedback remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience Replay (ACDER), which leverages (i) a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully and (ii) the dynamic initial stat…
Model-based reinforcement learning strategies are believed to exhibit more significant sample complexity than model-free strategies to control dynamical systems,such as quadcopters.This belief that Model-based strategies that involve the use of well-trained neural networks for making such high-level decisions always give better performance can be dispelled by making use of Model-free policy search methods.This paper proposes the use of a model-free random searching strategy,called Augmented Random Search(ARS),which is a better and faster approach of linear policy training for continuous contro…
Amazon Inc.（亚马逊公司）；Salesforce Research（赛富时研究院）；University of Pittsburgh（匹兹堡大学）；Chinese University of Hong Kong（香港中文大学）
Keyphrase generation is the task of automatically predicting keyphrases given a piece of long text. Despite its recent flourishing, keyphrase generation on non-English languages haven't been vastly investigated. In this paper, we call attention to a new setting named multilingual keyphrase generation and we contribute two new datasets, EcommerceMKP and AcademicMKP, covering six languages. Technically, we propose a retrieval-augmented method for multilingual keyphrase generation to mitigate the data shortage problem in non-English languages. The retrieval-augmented model leverages keyphrase ann…
