What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks
Shandong University(山东大学);University of Electronic Science and Technology of China(电子科技大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Tool learning methods have enhanced the ability of large language models (LLMs) to interact with real-world applications. Many existing works fine-tune LLMs or design prompts to enable LLMs to select appropriate tools and correctly invoke them to meet user requirements. However, it is observed in previous works that the performance of tool learning varies from tasks, datasets, training settings, and algorithms. Without understanding the impact of these factors, it can lead to inconsistent results, inefficient model deployment, and suboptimal tool utilization, ultimately hindering the practical integration and scalability of LLMs in real-world scenarios. Therefore, in this paper, we explore the impact of both internal and external factors on the performance of tool learning frameworks. Through extensive experiments on two benchmark datasets, we find several insightful conclusions for future work, including the observation that LLMs can benefit significantly from increased trial and exploration. We believe our empirical study provides a new perspective for future tool learning research.
What Affects the Effective Depth of Large Language Models?
Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);Department of Computer Science, Massachusetts Institute of Technology(计算机科学系,麻省理工学院)
What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University(计算机科学学院,机器学习与产业智能工程研究中心,四川大学);School of Computer and Information Engineering, Xiamen University of Technology(计算机与信息工程学院,厦门理工大学);Department of Electrical and Computer Engineering, University of Hong Kong(电气与计算机工程系,香港大学);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
What Factors Affect LLMs and RLLMs in Financial Question Answering?
School of Computer Science and Engineering, Macau University of Science and Technology, China(澳门科技大学计算机科学与工程学院);SKLPlanets, Macau University of Science and Technology, China(澳门科技大学月球与行星科学国家重点实验室);School of Economics, Anhui University, China(安徽大学经济学院);School of Energy and Power Engineering, Huazhong University of Science and Technology, China(华中科技大学能源与动力工程学院)
What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks