Researchers and practitioners have recently reframed powerful Large Language Models (LLMs) as agents, enabling them to automate complex tasks largely via the use of specialized functions. To facilitate the development of LLM agents, we present a novel paradigm of training LLM agents without modifying the LLM weights, which is particularly useful when the LLMs are difficult or inaccessible for modifications. Inspired by how humans continuously forge tools to adapt to real-world tasks, rather than change our biological structure to fit a static set of tools, we propose to progressively forge agent's functions to better solve the downstream tasks instead of modifying the LLM weights. By treating the functions as learnable `agent parameters' and leveraging the fundamental idea of model training in artificial intelligence, we develop AgentOptimizer that employs the LLM to update agents' functions and devise an agent training algorithm with two strategies, roll-back, and early-stop, to streamline the training process. With extensive experiments, we showcase that the agent training paradigm could significantly improve the performance of representative LLM agents in various downstream tasks. We also study the behavior of the agent training regarding aspects like the learning curve and domain transferability.
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other
University of Electronic Science and Technology of China(电子科技大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Beijing Normal University(北京师范大学);Tianjin University(天津大学)
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
The Hong Kong University of Science and Technology(香港科技大学);City University of Hong Kong(香港城市大学);University of Electronic Science and Technology of China(电子科技大学)
Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization
School of Computer Science and Engineering, Southeast University, Nanjing, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其交叉应用国家重点实验室)
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);China Nuclear Power Engineering Company Ltd(中国核电工程有限公司);IEG, Tencent(腾讯互动娱乐事业群);School of Computer Science, Peking University(北京大学计算机学院)
Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL
Tsinghua University(清华大学);University of Cambridge(剑桥大学);Eindhoven University of Technology(埃因霍温理工大学);King’s College London(伦敦国王学院);Tencent Robotics X(腾讯机器人实验室)
Environment Transformer and Policy Optimization for Model-Based Offline Reinforcement Learning
The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangdong Provincial Key Lab of Integrated Communication, Sensing and Computation for Ubiquitous Internet of Things(广东省泛在物联网融合通信感知计算重点实验室)