Tool-calling agents are increasingly deployed in real-world customer-facing workflows. Yet most studies on tool-calling agents focus on idealized settings with general, fixed, and well-specified tasks. In real-world applications, user requests are often (1) ambiguous, (2) changing over time, or (3) infeasible due to policy constraints, and training and evaluation data that cover these diverse, complex interaction patterns remain under-represented. To bridge the gap, we present Trajectory2Task, a verifiable data generation pipeline for studying tool use at scale under three realistic user scenarios: ambiguous intent, changing intent, and infeasible intents. The pipeline first conducts multi-turn exploration to produce valid tool-call trajectories. It then converts these trajectories into user-facing tasks with controlled intent adaptations. This process yields verifiable task that support closed-loop evaluation and training. We benchmark seven state-of-the-art LLMs on the generated complex user scenario tasks and observe frequent failures. Finally, using successful trajectories obtained from task rollouts, we fine-tune lightweight LLMs and find consistent improvements across all three conditions, along with better generalization to unseen tool-use domains, indicating stronger tool-calling ability.
Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统认知与决策智能重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents
Shenyang Institute of Computing Technology, Chinese Academy of Sciences(中国科学院沈阳计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)(教育部计算电力网络与信息安全重点实验室,山东计算机科学中心(济南国家超算中心),齐鲁工业大学(山东省科学院))
FASTRIC: Prompt Specification Language for Verifiable LLM Interactions
Department of Civil and Environmental Engineering(土木与环境工程系);California Institute for Telecommunications and Information Technology(电信与信息科技学院);Institute of Transportation Studies(交通研究学院);University of California, Irvine, CA 92697-3600(加州大学伊维德分校)
Beyond Ideal Instruction: A Comprehensive Framework for Evaluating LLMs in Realistic Interactions
Department of Data Science, City University of Hong Kong(香港城市大学数据科学系);Hong Kong Institute of AI for Science, City University of Hong Kong(香港城市大学人工智能科学研究院);Li Auto Inc.(Li汽车公司);Beijing University of Posts and Telecommunications(北京邮电大学);Independent Researcher(独立研究员)
Conv-to-Bench: Evaluating Language Models Via User-Assistant Dialogues In Code Tasks
Institute of Mathematics and Computer Science, University of São Paulo(圣保罗大学数学与计算机科学学院);Institute of Informatics, Federal University of Goiás(戈亚斯联邦大学信息学院);HUG Labs(HUG实验室);Advanced Knowledge Center for Immersive Technologies (AKCIT)(沉浸式技术高级知识中心)
IDALC: A Semi-Supervised Framework for Intent Detection and Active Learning based Correction
Department of Computer Science and Engineering, IIT Kharagpur(计算机科学与工程系,印度IIT Kharagpur);Department of Computer Science, Technische Universität Darmstadt(计算机科学系,德国Technische Universität Darmstadt);Adobe Research, Bangalore(Adobe Research,印度班加罗尔)
Beyond Static Endpoints: Tool Programs as an Interface for Flexible Agentic Web Services
School of Computer Science, Peking University, Beijing, China;School of Software \& Microelectronics, Peking University, Beijing, China;Institute for Artificial Intelligence, Peking University, Beijing, China