Recent advancements in Large Language Models (LLMs) have demonstrated exceptional capabilities in natural language understanding and generation. While these models excel in general complex reasoning tasks, they still face challenges in mathematical problem-solving and logical reasoning. To address these limitations, researchers have explored function calling abilities, allowing LLMs to execute provided functions and utilize their outputs for task completion. However, concentrating on specific tasks can be very inefficient for large-scale LLMs to be used, because of the expensive cost of training and inference stages they need in terms of computational resources. This study introduces a novel framework for training smaller language models in function calling, focusing on specific logical and mathematical reasoning tasks. The approach aims to improve performances of small-scale models for these tasks using function calling, ensuring a high level of accuracy. Our framework employs an agent that, given a problem and a set of callable functions, queries the LLM by injecting a description and examples of the usable functions into the prompt and managing their calls in a step-by-step reasoning chain. This process is used to create a dataset of correct and incorrect reasoning chain chat completions from a large-scale LLM. This dataset is used to train a smaller LLM using Reinforcement Learning from Human Feedback (RLHF), specifically employing the Direct Preference Optimization (DPO) technique. Experimental results demonstrate how the proposed approach balances the trade-off between model size and performance, improving the ability of function calling for reasoning tasks, in smaller models.
MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling
College of Health Science and Technology, Shanghai Jiao Tong University School of Medicine, Shanghai, China(上海交通大学医学院健康科学与技术学院);Fudan University, Shanghai, China(复旦大学);Shanghai Innovation Institute, Shanghai, China(上海创新研究院);Tsinghua University, Beijing, China(清华大学);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学);Ruijin Hospital, Shanghai Jiaotong University, Shanghai, China(上海交通大学瑞金医院)
Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents
University of Houston(休斯顿大学);University of Southern California(南加州大学);New York University(纽约大学);Texas A&M University(德克萨斯农工大学);University of California, San Diego(加州大学圣地亚哥分校)
Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning
Southeast University(东南大学);King’s College London(伦敦国王学院);The Alan Turing Institute(艾伦·图灵研究所);Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东科学院))
MARGE: Improving Math Reasoning for LLMs with Guided Exploration
Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China(清华大学交叉信息学院);Alibaba Group(阿里巴巴集团);Shanghai Qi Zhi Institute, Shanghai, China(上海启智研究院)
Improving Rationality in the Reasoning Process of Language Models through Self-playing Game
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Key Laboratory of Data Intelligence and Advanced Computing, Soochow University(苏州大学数据智能与先进计算重点实验室)
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning
Department of Electronic Engineering, Kyung Hee University(韩国庆熙大学电子工程系);School of Computer Science and Engineering, University of Electronic Science and Technology of China(中国电子科技大学计算机科学与工程学院);Henan Polytechnic University(河南理工大学);School of Computing, Kyung Hee University(韩国庆熙大学计算机学院)
RC-GRPO: Reward-Conditioned Group Relative Policy Optimization for Multi-Turn Tool Calling Agents
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(模式识别新实验室、多模态人工智能系统国家重点实验室、自动化研究所,中国科学院);School of Mathematics and Statistics(数学与统计学学院);Statistics, Lanzhou University(统计学,兰州大学);Shanghai Innovation Institute(上海创新研究院);Microsoft Research(微软研究院);Nanjing University(南京大学);Zhongguancun Academy(中关村学院)
Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China(国家新型软件技术重点实验室,南京大学,中国);School of Artificial Intelligence, Nanjing University, China(人工智能学院,南京大学,中国);School of Intelligence Science and Technology, Nanjing University, China(智能科学与技术学院,南京大学,中国)
Improving Context Fidelity via Native Retrieval-Augmented Reasoning
DIRO, Université de Montréal(蒙特利尔大学计算机科学与运筹学系);MetaGPT;Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);McGill University(麦吉尔大学);Yale University(耶鲁大学);Canada CIFAR AI Chair(加拿大CIFAR人工智能教席)
Direct Value Optimization: Improving Chain-of-Thought Reasoning in LLMs with Refined Values
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究所);University College London(伦敦大学学院)