Large language models (LLMs) show an innate skill for solving language based tasks. But insights have suggested an inability to adjust for information or task-solving skills becoming outdated, as their knowledge, stored directly within their parameters, remains static in time. Tool use helps by offloading work to systems that the LLM can access through an interface, but LLMs that use them still must adapt to nonstationary environments for prolonged use, as new tools can emerge and existing tools can change. Nevertheless, tools require less specialized knowledge, therefore we hypothesize they are better suited for continual learning (CL) as they rely less on parametric memory for solving tasks and instead focus on learning when to apply pre-defined tools. To verify this, we develop a synthetic benchmark and follow this by aggregating existing NLP tasks to form a more realistic testing scenario. While we demonstrate scaling model size is not a solution, regardless of tool usage, continual learning techniques can enable tool LLMs to both adapt faster while forgetting less, highlighting their potential as continual learners.
LLMs Can Evolve Continually on Modality for X-Modal Reasoning
Dalian University of Technology（大连理工大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology（香港科技大学）
Continual learning usually assumes the incoming data are fully labeled, which might not be applicable in real applications. In this work, we consider semi-supervised continual learning (SSCL) that incrementally learns from partially labeled data. Observing that existing continual learning methods lack the ability to continually exploit the unlabeled data, we propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN), which continually passes the learned data distribution to the classifier. I…
LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions
University of Washington（华盛顿大学）；Pioneer Centre for Artificial Intelligence（先锋人工智能中心）；University of Copenhagen（哥本哈根大学）；Princeton University（普林斯顿大学）；Allen Institute for Artificial Intelligence (Ai2)（艾伦人工智能研究所（Ai2））
ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada（电气与计算机工程系，不列颠哥伦比亚大学）；Department of Computer Science and Engineering, Southern University of Science and Technology, China（计算机科学与工程系，南方科技大学）
MetaToolAgent: Towards Generalizable Tool Usage in LLMs through Meta-Learning
College of Informatics, Huazhong Agricultural University（华中农业大学信息学院）；Industrial AI Research Centre, University of South Australia（南澳大利亚大学工业AI研究中心）；School of Computer Science, Wuhan University（武汉大学计算机学院）
Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Huawei Technologies Co., Ltd（华为技术有限公司）；The University of Hong Kong（香港大学）；Peng Cheng Laboratory, Shenzhen（鹏城实验室）
Joint Continual Learning of Local Language Models and Cloud Offloading Decisions with Budget Constraints
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN（电子与计算机工程学院，普渡大学）；Department of Computer Science, University of Exeter, UK（计算机科学系，埃克塞特大学）
Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Alibaba Group（阿里巴巴集团）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）
Digger: Detecting Copyright Content Mis-usage in Large Language Model Training
Beijing University of Posts and Telecommunications（北京邮电大学）；Nanyang Technological University（南洋理工大学）；Huazhong University of Science and Technology（华中科技大学）；The University of New South Wales（新南威尔士大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
