In this paper, we fully answer the above question through a key algebraic condition on graph functions, called \textit{permutation compatibility} , that relates permutations of weights and features of the graph to functional constraints. We prove that: (i) a GNN, as a graph function, is necessarily permutation compatible; (ii) conversely, any permutation compatible function, when restricted on input graphs with distinct node features, can be generated by a GNN; (iii) for arbitrary node features (not necessarily distinct), a simple feature augmentation scheme suffices to generate a permutation compatible function by a GNN; (iv) permutation compatibility can be verified by checking only quadratically many functional constraints, rather than an exhaustive search over all the permutations; (v) GNNs can generate \textit{any} graph function once we augment the node features with node identities, thus going beyond graph isomorphism and permutation compatibility. The above characterizations pave the path to formally study the intricate connection between GNNs and other algorithmic procedures on graphs. For instance, our characterization implies that many natural graph problems, such as min-cut value, max-flow value, max-clique size, and shortest path can be generated by a GNN using a simple feature augmentation. In contrast, the celebrated Weisfeiler-Lehman graph-isomorphism test fails whenever a permutation compatible function with identical features cannot be generated by a GNN. At the heart of our analysis lies a novel representation theorem that identifies basis functions for GNNs. This enables us to translate the properties of the target graph function into properties of the GNN's aggregation function.
英文摘要：
In this paper, we fully answer the above question through a key algebraic condition on graph functions, called \textit{permutation compatibility}, that relates permutations of weights and features of the graph to functional constraints. We prove that: (i) a GNN, as a graph function, is necessarily permutation compatible; (ii) conversely, any permutation compatible function, when restricted on input graphs with distinct node features, can be generated by a GNN; (iii) for arbitrary node features (not necessarily distinct), a simple feature augmentation scheme suffices to generate a permutation compatible function by a GNN; (iv) permutation compatibility can be verified by checking only quadratically many functional constraints, rather than an exhaustive search over all the permutations; (v) GNNs can generate \textit{any} graph function once we augment the node features with node identities, thus going beyond graph isomorphism and permutation compatibility. The above characterizations pave the path to formally study the intricate connection between GNNs and other algorithmic procedures on graphs. For instance, our characterization implies that many natural graph problems, such as min-cut value, max-flow value, max-clique size, and shortest path can be generated by a GNN using a simple feature augmentation. In contrast, the celebrated Weisfeiler-Lehman graph-isomorphism test fails whenever a permutation compatible function with identical features cannot be generated by a GNN. At the heart of our analysis lies a novel representation theorem that identifies basis functions for GNNs. This enables us to translate the properties of the target graph function into properties of the GNN's aggregation function.
What Planning Problems Can A Relational Neural Network Solve?
MIT（麻省理工学院）；MIT Computer Science & Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）；MIT Department of Brain and Cognitive Sciences（麻省理工学院脑与认知科学系）；Center for Brains, Minds and Machines（脑、心智与机器中心）
Massachusetts Institute of Technology (MIT)（麻省理工学院）；University of Maryland（马里兰大学）；Institute for Advanced Study (IAS)（普林斯顿高等研究院）；National Institute of Informatics (NII)（国立信息学研究所）
Measuring what Really Matters: Optimizing Neural Networks for TinyML
ETH Zürich（苏黎世联邦理工学院）；RWTH Aachen（亚琛工业大学）
随着廉价的计算和存储资源的激增，神经网络（NN）在架构和计算复杂性方面经历了前所未有的增长。将NN引入资源受限的设备可以实现经济高效的部署、广泛的可用性和敏感数据的保存。这项工作解决了将机器学习引入MCU的挑战，我们专注于无处不在的ARM Cortex-M架构。优化方法、软件框架和MCU硬件架构对推理延迟和能耗等关键性能指标的详细影响和权衡，以前尚未针对TensorFlow Lite Micro等最先进的框架进行深入研究。我们发现，经验调查，衡量可感知的指标-性能体验的用户-是必不可少的，因为专门的指令和层类型的影响可能是微妙的。为此，我们提出了一个实现感知的设计作为一个具有成本效益的方法进行验证和基准测试。利用我们开发的工具链，我们演示了如何通过系统地优化NN以适应其目标应用场景来改进资源受限设备上的现有NN部署。
What Makes a Good Example? Modeling Exemplar Selection with Neural Network Representations
Department of Psychology, University of Southern California（心理学系，南加州大学）；Department of Statistics and Data Science, University of California, Los Angeles（统计与数据科学系，加州大学洛杉矶分校）
Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation
Peking University（北京大学）；Beijing Institute of Big Data Research（北京大数据研究院）；Stanford University（斯坦福大学）；Huazhong University of Science and Technology（华中科技大学）；Cornell University（康奈尔大学）
In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Washington（华盛顿大学）；Colorado State University（科罗拉多州立大学）；National Geospatial-Intelligence Agency（国家地理空间情报局）
Who Does What in Deep Learning? Multidimensional Game-Theoretic Attribution of Function of Neural Units
Institute of Computational Neuroscience（计算神经科学研究所）；University Medical Center Eppendorf（埃本多夫大学医学中心）；International Max Planck Research School on Cognitive Neuroimaging（认知神经成像国际马克斯·普朗克研究学校）；MRC Cognition and Brain Sciences Unit（MRC认知与脑科学单位）；Mila - Quebec Artificial Intelligence Institute（魁北克人工智能研究所）；Learning in Machines & Brains（机器与大脑学习）；Departments of Bioengineering and Neuroscience（生物工程与神经科学系）；Department of Health Sciences（健康科学系）
Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next
University of Naples Federico II（那不勒斯费德里科二世大学）；SISSA – International School for Advanced Studies（国际高等研究学院（SISSA））；University of Colorado Boulder（科罗拉多大学博尔德分校）
