Communication between multiple language model (LM) agents has been shown to scale up the reasoning ability of LMs. While natural language has been the dominant medium for inter-LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be otherwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via activations; concretely, we pause an LM $\textit{B}$'s computation at an intermediate layer, combine its current activation with another LM $\textit{A}$'s intermediate activation via some function $\textit{f}$, then pass $\textit{f}$'s output into the next layer of $\textit{B}$ and continue the forward pass till decoding is complete. This approach scales up LMs on new tasks with zero additional parameters and data, and saves a substantial amount of compute over natural language communication. We test our method with various functional forms $\textit{f}$ on two experimental setups--multi-player coordination games and reasoning benchmarks--and find that it achieves up to $27.0\%$ improvement over natural language communication across datasets with $<$$1/4$ the compute, illustrating the superiority and robustness of activations as an alternative "language" for communication between LMs.
Simulating the Emergence of Differential Case Marking with Communicating Neural-Network Agents
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University(西安交通大学电子与信息工程学院);Leiden Institute of Advanced Computer Science, Leiden University(莱顿大学高级计算机科学研究所);Center for Language and Cognition, University of Groningen(格罗宁根大学语言与认知中心)
Hey AI, Can You Solve Complex Tasks by Talking to Agents?
Allen Institute for AI(艾伦人工智能研究所)
为每个复杂任务从头开始训练巨型模型是资源和数据效率低下的。为了帮助开发能够利用现有系统的模型,我们提出了一个新的挑战:通过用自然语言与现有代理(或模型)进行通信来学习解决复杂任务。我们设计了一个综合基准,CommaQA,三个复杂的推理任务(显式,隐式,数值),旨在解决与现有的QA代理通信。例如,使用文本和表格QA代理来回答诸如“美国谁的标枪投掷最长?".我们表明,黑盒模型努力从头开始学习这个任务(准确率低于50%),即使访问每个代理的知识和黄金事实监督。相比之下,学习与代理沟通的模型优于黑盒模型,在给予黄金分解监督时达到100%的分数。然而,我们发现,通过与现有代理\emph{without relying on any auxiliary supervision or data}通信来学习解决复杂任务的挑战仍然非常难以捉摸。我们发布了CommaQA,以及一个组合泛化测试分割,以推进这一方向的研究。数据集和代码可在https://github.com/allenai/commaqa获得。
Towards Adaptive Mechanism Activation in Language Agent
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
Wuhan University(武汉大学);Hong Kong Baptist University(香港 Baptist大学);New Jersey Institute of Technology(新泽西理工学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation
School of Computing, University of Georgia(佐治亚大学计算机学院);Department of Biosystems Engineering and Soil Science, University of Tennessee(田纳西大学生物系统工程与土壤科学系);Department of Biomedical Informatics, University of Colorado School of Medicine(科罗拉多医学院生物医学信息学系);Department of Biomedical Engineering, New Jersey Institute of Technology(新泽西理工学院生物医学工程系);Department of Physics and Astronomy, The University of Georgia(佐治亚大学物理与天文学系)