Recent work in the multi-agent domain has shown the promise of Graph Neural Networks (GNNs) to learn complex coordination strategies. However, most current approaches use minor variants of a Graph Convolutional Network (GCN), which applies a convolution to the communication graph formed by the multi-agent system. In this paper, we investigate whether the performance and generalization of GCNs can be improved upon. We introduce ModGNN, a decentralized framework which serves as a generalization of GCNs, providing more flexibility. To test our hypothesis, we evaluate an implementation of ModGNN against several baselines in the multi-agent flocking problem. We perform an ablation analysis to show that the most important component of our framework is one that does not exist in a GCN. By varying the number of agents, we also demonstrate that an application-agnostic implementation of ModGNN possesses an improved ability to generalize to new environments.
Fine-Grained Model Merging via Modular Expert Recombination
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(数据科学与人工智能系,香港理工大学);College of Computer Science, Chongqing University(计算机科学学院,重庆大学)
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
McGill University(麦吉尔大学);Mila(Mila实验室);Microsoft(微软公司);Université de Montréal(蒙特利尔大学);ServiceNow(ServiceNow公司);Georgia Institute of Technology(佐治亚理工学院)
University of Copenhagen(哥本哈根大学);University of Montreal(蒙特利尔大学);Halmstad University(哈尔姆斯塔德大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing
University of Colorado Boulder(科罗拉多大学博尔德分校);National Institute of Mental Health(美国国家精神卫生研究所);Johannes Gutenberg University Mainz(美因茨约翰内斯·古腾堡大学);University of Chicago(芝加哥大学);Google DeepMind(谷歌DeepMind)
A2C: A Modular Multi-stage Collaborative Decision Framework for Human-AI Teams
Institute for Clarity in Documentation(文献清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(复杂系统认知与决策智能重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Information Flow Control in Machine Learning through Modular Model Architecture
Cornell University(康奈尔大学);University of Washington(华盛顿大学);FAIR at Meta(Meta FAIR实验室);Google DeepMind(谷歌DeepMind);Georgia Institute of Technology(佐治亚理工学院);Virginia Tech(弗吉尼亚理工大学);Pennsylvania State University(宾夕法尼亚州立大学);Intel(英特尔);NVIDIA(英伟达);Meta
Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess
Computer Science Department TU Darmstadt Darmstadt, Germany 0000-0001-5681-532X;Machine Learning Lab TU Darmstadt Darmstadt, Germany 0000-0002-9568-8965;Machine Learning Lab, Hessian Center for Artificial Intelligence (hessian.AI) TU Darmstadt Darmstadt, Germany 0000-0002-9400-0946;Computer Science Department TU Darmstadt Darmstadt, Germany 0009-0004-9082-9871;Machine Learning Lab, Hessian Center for Artificial Intelligence (hessian.AI), Centre for Cognitive Science, German Research Center for Artificial Intelligence (DFKI) TU Darmstadt Darmstadt, Germany 0000-0002-2873-9152
Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning
UNC Chapel Hill(北卡罗来纳大学教堂山分校)
多跳QA需要一个模型来连接分散在长上下文中的多个证据来回答问题。最近提出的HotpotQA(Yang等人,2018)数据集由体现四种不同的多跳推理范例(两个桥接实体设置、检查多个属性和比较两个实体)的问题组成,使得单个神经网络处理所有四个问题具有挑战性。在这项工作中,我们提出了一个可解释的,基于神经网络的自组装神经模块网络(Hu et al.,2017年,2018年)的多跳推理,在那里我们设计了四个新的模块(查找,重新定位,比较,NoOp)来执行独特类型的语言推理。基于一个问题,我们的布局控制器RNN动态地推断一系列推理模块来构建整个网络。从经验上讲,我们表明,我们的动态,多跳模块化网络实现了显着的改进,静态,单跳基线(定期和对抗性评估)。我们通过三个分析进一步证明了我们的模型的可解释性。首先,控制器可以将多跳问题软分解为多个单跳子问题,以促进主网络的组合推理行为。其次,控制器可以预测符合人类专家设计的布局的布局。最后,中间模块可以通过处理来自控制器的子问题来推断连接两个远距离定位的支持事实的实体。
Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
Zhejiang University(浙江大学);Shanghai Innovation Institute(上海创新研究院);ByteDance Inc.(字节跳动公司);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)
University of Oxford(牛津大学);Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);McGill University(麦吉尔大学);Google DeepMind(谷歌DeepMind)