TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
Xiaohongshu Inc.（小红书公司）；School of Computer Science, Fudan University（复旦大学计算机学院）；Dalian University of Technology（大连理工大学）；Peking University（北京大学）；Wuhan University（武汉大学）
As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce \textbf{TRIP-Bench}, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose \textbf{GTPO}, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training.
University of Washington（华盛顿大学）；Facebook AI Research（Facebook 人工智能研究院）；Princeton University（普林斯顿大学）
Existing work in language grounding typically study single environments. How do we build unified models that apply across multiple environments? We propose the multi-environment Symbolic Interactive Language Grounding benchmark (SILG), which unifies a collection of diverse grounded language learning environments under a common interface. SILG consists of grid-world environments that require generalization to new dynamics, entities, and partially observed worlds (RTFM, Messenger, NetHack), as well as symbolic counterparts of visual worlds that require interpreting rich natural language with res…
ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training
National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家实验室，南京大学）；School of Artificial Intelligence, Nanjing University, Nanjing, China（人工智能学院，南京大学，中国南京）；Institute of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, China（计算机科学与技术学院，哈尔滨工业大学（深圳），中国深圳）；School of Statistics, East China Normal University, Shanghai, China（统计学学院，华东师范大学，中国上海）
Performing simple household tasks based on language directives is very natural to humans, yet it remains an open challenge for AI agents. The 'interactive instruction following' task attempts to make progress towards building agents that jointly navigate, interact, and reason in the environment at every step. To address the multifaceted problem, we propose a model that factorizes the task into interactive perception and action policy streams with enhanced components and name it as MOCA, a Modular Object-Centric Approach. We empirically validate that MOCA outperforms prior arts by significant m…
Stanford University（斯坦福大学）；Robotics at Google（谷歌机器人部门）
We present Interactive Gibson Benchmark, the first comprehensive benchmark for training and evaluating Interactive Navigation: robot navigation strategies where physical interaction with objects is allowed and even encouraged to accomplish a task. For example, the robot can move objects if needed in order to clear a path leading to the goal location. Our benchmark comprises two novel elements: 1) a new experimental setup, the Interactive Gibson Environment (iGibson 0.5), which simulates high fidelity visuals of indoor scenes, and high fidelity physical dynamics of the robot and common objects …
PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors
Center of Excellence for Generative AI, KAUST（生成人工智能卓越中心，KAUST）；The Swiss AI Lab, IDSIA-USI/SUPSI（瑞士人工智能实验室，IDSIA-USI/SUPSI）；NNAISENSE Principia-AI/PhysGym（NNAISENSE 原初人工智能/PhysGym）
This paper presents a new natural language processing task - Actionable Entities Recognition (AER) - recognition of entities that protagonists could interact with for further plot development. Though similar to classical Named Entity Recognition (NER), it has profound differences. In particular, it is crucial for interactive fiction, where the agent needs to detect entities that might be useful in the future. We also discuss if AER might be further helpful for the systems dealing with narrative processing since actionable entities profoundly impact the causal relationship in a story. We valida…
ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents
University of North Carolina at Chapel Hill（北卡罗来纳大学切里波因特分校）；Stanford University（斯坦福大学）；University of California, Berkeley（加州大学伯克利分校）；University of Southern California（南加州大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Xiamen University Malaysia（马来西亚厦门大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping
The Chinese University of Hong Kong, China（香港中文大学）；Singapore Management University, Singapore（新加坡管理学院）；Renmin University of China, China（中国人民大学）；Tsinghua University, China（清华大学）；Southwest University, China（西南大学）
Words to World: Benchmarking LLMs in Interactive Physical Environments
Michigan State University（密歇根州立大学）；Zhejiang University（浙江大学）；Microsoft（微软）；University of Rochester（罗切斯特大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
Shenzhen Institute of Advanced Technology Chinese Academy of Sciences（深圳先进技术研究院中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Dalian University of Technology（大连理工大学）；UNSW Sydney（悉尼大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）
Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles
Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；College of Energy Engineering, Zhejiang University（浙江大学能源工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Didi Chuxing（滴滴出行）
University of Texas Austin（德克萨斯大学奥斯汀分校）；California Institute of Technology（加州理工学院）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Microsoft Research（微软研究院）；Northwestern University（西北大学）；University of Cambridge（剑桥大学）
Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives
University of Macau（澳门大学）；Westlake University（西湖大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Cambridge（剑桥大学）；University of Aberdeen（阿伯丁大学）
IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）智能科学与工程学院）；Zhejiang University（浙江大学）；State Key Lab of CAD&CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）
