In this work, we review research studies that combine Reinforcement Learning (RL) and Large Language Models (LLMs), two areas that owe their momentum to the development of deep neural networks. We propose a novel taxonomy of three main classes based on the way that the two model types interact with each other. The first class, RL4LLM, includes studies where RL is leveraged to improve the performance of LLMs on tasks related to Natural Language Processing. L4LLM is divided into two sub-categories depending on whether RL is used to directly fine-tune an existing LLM or to improve the prompt of the LLM. In the second class, LLM4RL, an LLM assists the training of an RL model that performs a task that is not inherently related to natural language. We further break down LLM4RL based on the component of the RL training framework that the LLM assists or replaces, namely reward shaping, goal generation, and policy function. Finally, in the third class, RL+LLM, an LLM and an RL agent are embedded in a common planning framework without either of them contributing to training or fine-tuning of the other. We further branch this class to distinguish between studies with and without natural language feedback. We use this taxonomy to explore the motivations behind the synergy of LLMs and RL and explain the reasons for its success, while pinpointing potential shortcomings and areas where further research is needed, as well as alternative methodologies that serve the same goal.
Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models
CISPA Helmholtz Center for Information Security(CISPA赫尔姆霍茨信息安全中心);Microsoft(微软);University College London(伦敦大学学院);Mila, University of Montreal(蒙特利尔大学Mila)
A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review
Federation University(联邦大学);Deakin University(迪肯大学);Universidad Central de Chile(智利中央大学);University of Alberta(阿尔伯塔大学);Action Research Associates(行动研究协会)
LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios
Beijing Jiaotong University(北京交通大学);Lancaster University(兰卡斯特大学);Max Planck Institute for Informatics, Saarland Informatics Campus(马克斯·普朗克研究所(信息学)萨尔兰州信息学校区);University of Electronic Science and Technology of China(电子科技大学)
Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents
Department of Computer Science, New York University, CIMS, New York, USA(纽约大学计算机科学系,CIMS,纽约,美国);Department of Computer Science, Cleveland State University, Cleveland, USA(克利夫兰州立大学计算机科学系,克利夫兰,美国)
Tool Learning with Large Language Models: A Survey
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Baidu Inc.(百度公司);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
Bridge-AI Lab(Bridge-AI实验室);University of Central Florida(中佛罗里达大学);Auburn University(奥本大学);Wichita State University(威奇托州立大学);Gustavus Adolphus College(古斯塔夫斯阿道夫学院)
School of Advanced Interdisciplinary Sciences, UCAS(UCAS交叉学科研究院);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院);Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models
Center for Astrophysics, Harvard & Smithsonian(哈佛与史密森尼天体物理中心);Washington University in St. Louis(圣路易斯华盛顿大学);Australian National University(澳大利亚国立大学);European Commission, Joint Research Centre (JRC)(欧盟委员会联合研究中心);Intelligent Internet Inc.(智能互联网公司);University of Arizona(亚利桑那大学);Indian Institute of Technology, Delhi(印度德里理工学院);Institute of Fundamental Technological Research, Polish Academy of Sciences(波兰科学院基础技术研究所);Walgreens Boots Alliance AI Lab(沃尔格林博姿联盟人工智能实验室);Columbia University(哥伦比亚大学);UniverseTBD;Oak Ridge National Laboratory(橡树岭国家实验室);Instituto de Astrofísica de Canarias(加那利群岛天体物理研究所);European Space Agency(欧洲空间局);Modulos AG(Modulos AG公司);Stanford University(斯坦福大学)