Learning to use tools to solve a variety of tasks is an innate ability of humans and has been observed of animals in the wild. However, the underlying mechanisms that are required to learn to use tools are abstract and widely contested in the literature. In this paper, we study tool use in the context of reinforcement learning and propose a framework for analyzing generalization inspired by a classic study of tool using behavior, the trap-tube task. Recently, it has become common in reinforcement learning to measure generalization performance on a single test set of environments. We instead propose transfers that produce multiple test sets that are used to measure specified types of generalization, inspired by abilities demonstrated by animal and human tool users. The source code to reproduce our experiments is publicly available at https://github.com/fomorians/gym_tool_use.
Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspective
Gaoling School of Artificial Intelligence(北京理工大学人工智能学院);Engineering Research Center of Next-Generation Intelligent Search and Recommendation(下一代智能搜索与推荐工程研究中心)
A Decomposition Perspective to Long-context Reasoning for LLMs
Large Language Model Department, Tencent(腾讯大语言模型部门);Xi'an Jiaotong University(西安交通大学);Tencent(腾讯);Fudan University(复旦大学);City University of Hong Kong(香港城市大学)
Generalization of RLVR Using Causal Reasoning as a Testbed
Johns Hopkins University(约翰霍普金斯大学);University of Maryland, College Park(马里兰大学学院公园分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research Asia(微软亚洲研究院);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Institute of Foundation Models(基础模型研究院);University of Michigan(密歇根大学)