Neural Algorithmic Reasoning is an emerging area of machine learning which seeks to infuse algorithmic computation in neural networks, typically by training neural models to approximate steps of classical algorithms. In this context, much of the current work has focused on learning reachability and shortest path graph algorithms, showing that joint learning on similar algorithms is beneficial for generalisation. However, when targeting more complex problems, such similar algorithms become more difficult to find. Here, we propose to learn algorithms by exploiting duality of the underlying algorithmic problem. Many algorithms solve optimisation problems. We demonstrate that simultaneously learning the dual definition of these optimisation problems in algorithmic learning allows for better learning and qualitatively better solutions. Specifically, we exploit the max-flow min-cut theorem to simultaneously learn these two algorithms over synthetically generated graphs, demonstrating the effectiveness of the proposed approach. We then validate the real-world utility of our dual algorithmic reasoner by deploying it on a challenging brain vessel classification task, which likely depends on the vessels' flow properties. We demonstrate a clear performance gain when using our model within such a context, and empirically show that learning the max-flow and min-cut algorithms together is critical for achieving such a result.
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms. Several important works have investigated whether neural networks can effectively reason like algorithms, typically by learning to execute them. The common trend in the area, however, is to generate targeted kinds of algorithmic data to evaluate specific hypotheses, making results hard to transfer across publications, and increasing the barrier of entry. To consolidate progress and work towards unified evaluation, we propose the CLRS Algorit…
Université de Montréal（蒙特利尔大学）；Mila（米拉人工智能研究所）；DeepMind；HEC Montréal（蒙特利尔高等商学院）
Learning to execute algorithms is a fundamental problem that has been widely studied. Prior work~\cite{veli19neural} has shown that to enable systematic generalisation on graph algorithms it is critical to have access to the intermediate steps of the program/algorithm. In many reasoning tasks, where algorithmic-style reasoning is important, we only have access to the input and output examples. Thus, inspired by the success of pre-training on similar tasks or data in Natural Language Processing (NLP) and Computer Vision, we set out to study how we can transfer algorithmic reasoning knowledge. S…
Mila（米拉研究所）；Université de Montreal（蒙特利尔大学）；Google Research（谷歌研究院）
Large language models (LLMs) have shown increasing in-context learning capabilities through scaling up model and data size. Despite this progress, LLMs are still unable to solve algorithmic reasoning problems. While providing a rationale with the final answer has led to further improvements in multi-step reasoning problems, Anil et al. 2022 showed that even simple algorithmic reasoning tasks such as parity are far from solved. In this work, we identify and study four key stages for successfully teaching algorithmic reasoning to LLMs: (1) formulating algorithms as skills, (2) teaching multiple …
AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models
Microsoft Research NYC（微软研究院纽约分部）；Center for Theoretical Neuroscience（理论神经科学中心）；Department of Psychology（心理学系）；University of California Los Angeles（加州大学洛杉矶分校）；Institute for Pure and Applied Mathematics（纯粹与应用数学研究所）；Emory University（埃默里大学）；Rice University（里士满大学）；Mount Holyoke College（马里兰霍克学院）；Technische Universität Berlin（柏林技术大学）；BIFOLD-Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another. We argue that algorithms possess fundamentally different qualities to deep learning methods, and this strongly suggests that, were deep learning methods better able to mimic algorithms, generalisation of the sort seen with algorithms would become possible with deep learning -- something far out of the reach of current machine learning methods. Furthermore, by representing elements in a continuous space o…
University of British Columbia（不列颠哥伦比亚大学）；Google Research（谷歌研究院）
In this paper, we study the OOD generalization of neural algorithmic reasoning tasks, where the goal is to learn an algorithm (e.g., sorting, breadth-first search, and depth-first search) from input-output pairs using deep neural networks. First, we argue that OOD generalization in this setting is significantly different than common OOD settings. For example, some phenomena in OOD generalization of image classifications such as \emph{accuracy on the line} are not observed here, and techniques such as data augmentation methods do not help as assumptions underlying many augmentation techniques a…
BITS Pilani, K.K. Birla Goa Campus（印度比尔拉理工学院果阿校区）；TCS Research, New Delhi（塔塔咨询服务公司研究院）
We consider a class of visual analogical reasoning problems that involve discovering the sequence of transformations by which pairs of input/output images are related, so as to analogously transform future inputs. This program synthesis task can be easily solved via symbolic search. Using a variation of the `neural analogical reasoning' approach of (Velickovic and Blundell 2021), we instead search for a sequence of elementary neural network transformations that manipulate distributed representations derived from a symbolic space, to which input images are directly encoded. We evaluate the exte…
We present a new method for scaling automatic configuration of computer networks. The key idea is to relax the computationally hard search problem of finding a configuration that satisfies a given specification into an approximate objective amenable to learning-based techniques. Based on this idea, we train a neural algorithmic model which learns to generate configurations likely to (fully or partially) satisfy a given specification under existing routing protocols. By relaxing the rigid satisfaction guarantees, our approach (i) enables greater flexibility: it is protocol-agnostic, enables cro…
MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning
Pacific Northwest National Laboratory, Richland, WA（太平洋西北国家实验室）；Halıcıoğlu Data Science Institute, University of California, San Diego, San Diego, CA（哈利奇奥格鲁数据科学研究所，加州大学圣地亚哥分校）；Department of Computer and Information Science, University of Pennsylvania, Pennsylvaina, PA（计算机与信息科学系，宾夕法尼亚大学）；Department of Mathematics, University of Washington, Seattle, WA（数学系，华盛顿大学）
Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems. This has numerous benefits, such as allowing applications of algorithms when preconditions are not satisfied, or reusing learned models when sufficient training data is not available or can't be generated. Unfortunately, a key hindrance of these approaches is their lack of explainability, since GNNs are black-box models that cannot be interpreted directly. In this work, we address this limitation by applying existing work on concept-based explana…
Deep learning has excelled on complex pattern recognition tasks such as image classification and object recognition. However, it struggles with tasks requiring nontrivial reasoning, such as algorithmic computation. Humans are able to solve such tasks through iterative reasoning -- spending more time thinking about harder tasks. Most existing neural networks, however, exhibit a fixed computational budget controlled by the neural network architecture, preventing additional computational processing on harder tasks. In this work, we present a new framework for iterative reasoning with neural netwo…
Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation
IIIT Hyderabad（印度信息技术研究所海得拉巴分校）；ELLIS Institute Tübingen（埃利斯研究所蒂宾根分校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（蒂宾根人工智能中心）；University of Tübingen（蒂宾根大学）
Empowering LLMs in Decision Games through Algorithmic Data Synthesis
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Beihang University（北京航空航天大学）；The Chinese University of Hong Kong（香港中文大学）；Novosibirsk State University（新西伯利亚国立大学）
Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code
University of Science and Technology of China（中国科学技术大学）；Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Microsoft Research Asia（微软亚洲研究院）
Dual learning has been successfully applied in many machine learning applications including machine translation, image-to-image transformation, etc. The high-level idea of dual learning is very intuitive: if we map an $x$ from one domain to another and then map it back, we should recover the original $x$. Although its effectiveness has been empirically verified, theoretical understanding of dual learning is still very limited. In this paper, we aim at understanding why and when dual learning works. Based on our theoretical analysis, we further extend dual learning by introducing more related m…
University of Cambridge（剑桥大学）；DeepMind（深度思维）；Mila（米拉人工智能研究所）；Université de Montréal（蒙特利尔大学）
Neural algorithmic reasoning studies the problem of learning algorithms with neural networks, especially with graph architectures. A recent proposal, XLVIN, reaps the benefits of using a graph neural network that simulates the value iteration algorithm in deep reinforcement learning agents. It allows model-free planning without access to privileged information about the environment, which is usually unavailable. However, XLVIN only supports discrete action spaces, and is hence nontrivially applicable to most tasks of real-world interest. We expand XLVIN to continuous action spaces by discretiz…
Recent advances in neural algorithmic reasoning with graph neural networks (GNNs) are propped up by the notion of algorithmic alignment. Broadly, a neural network will be better at learning to execute a reasoning task (in terms of sample complexity) if its individual components align well with the target algorithm. Specifically, GNNs are claimed to align with dynamic programming (DP), a general problem-solving strategy which expresses many polynomial-time algorithms. However, has this alignment truly been demonstrated and theoretically quantified? Here we show, using methods from category theo…
University of Maryland（马里兰大学）；University of Chicago（芝加哥大学）；New York University（纽约大学）
Machine learning systems perform well on pattern matching tasks, but their ability to perform algorithmic or logical reasoning is not well understood. One important reasoning capability is algorithmic extrapolation, in which models trained only on small/simple reasoning problems can synthesize complex strategies for large/complex problems at test time. Algorithmic extrapolation can be achieved through recurrent systems, which can be iterated many times to solve difficult reasoning problems. We observe that this approach fails to scale to highly complex problems because behavior degenerates whe…
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
Courant Institute of Mathematical Sciences, New York University（纽约大学应用数学科学研究所）；Google Research（谷歌研究）；Meta AI；Bar-Ilan University（巴伊兰大学）；Department of Bio-Medical Engineering, Edmond J. Safra Center for Bioinformatics, Tel-Aviv University（生物医学工程系，埃德蒙·J·萨法中心，特拉维夫大学）；Tel Aviv University（特拉维夫大学）
Massachusetts Institute of Technology (MIT)（麻省理工学院）；University of Maryland（马里兰大学）；Institute for Advanced Study (IAS)（普林斯顿高等研究院）；National Institute of Informatics (NII)（国立信息学研究所）
Neural networks have succeeded in many reasoning tasks. Empirically, these tasks require specialized network structures, e.g., Graph Neural Networks (GNNs) perform well on many such tasks, but less structured networks fail. Theoretically, there is limited understanding of why and when a network structure generalizes better than others, although they have equal expressive power. In this paper, we develop a framework to characterize which reasoning tasks a network can learn well, by studying how well its computation structure aligns with the algorithmic structure of the relevant reasoning proces…
Department of Computer Science and Technology（计算机科学与技术系）；University of Cambridge（剑桥大学）
While modern deep neural architectures generalise well when test data is sampled from the same distribution as training data, they fail badly for cases when the test data distribution differs from the training distribution even along a few dimensions. This lack of out-of-distribution generalisation is increasingly manifested when the tasks become more abstract and complex, such as in relational reasoning. In this paper we propose a neuroscience-inspired inductive-biased module that can be readily amalgamated with current neural network architectures to improve out-of-distribution (o.o.d) gener…
Scaling Online Complex Event Detection with Synthetic Supervision and Mamba-Based Neural Algorithmic Reasoning
University of California, Los Angeles（加州大学洛杉矶分校）；University of Brescia（布雷西亚大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；US Army DEVCOM Army Research Laboratory（美国陆军 DEVCOM 军事研究实验室）；Amazon（亚马逊）
