Few-shot learning with sequence-processing neural networks (NNs) has recently attracted a new wave of attention in the context of large language models. In the standard N-way K-shot learning setting, an NN is explicitly optimised to learn to classify unlabelled inputs by observing a sequence of NK labelled examples. This pressures the NN to learn a learning algorithm that achieves optimal performance, given the limited number of training examples. Here we study an auxiliary loss that encourages further acceleration of few-shot learning, by applying recently proposed bootstrapped meta-learning to NN few-shot learners: we optimise the K-shot learner to match its own performance achievable by observing more than NK examples, using only NK examples. Promising results are obtained on the standard Mini-ImageNet dataset. Our code is public.
Meta-learning empowers artificial intelligence to increase its efficiency by learning how to learn. Unlocking this potential involves overcoming a challenging meta-optimisation problem. We propose an algorithm that tackles this problem by letting the meta-learner teach itself. The algorithm first bootstraps a target from the meta-learner, then optimises the meta-learner by minimising the distance to that target under a chosen (pseudo-)metric. Focusing on meta-learning with gradients, we establish conditions that guarantee performance improvements and show that the metric can control meta-optim…
This paper considers Safe Policy Improvement (SPI) in Batch Reinforcement Learning (Batch RL): from a fixed dataset and without direct access to the true environment, train a policy that is guaranteed to perform at least as well as the baseline policy used to collect the data. Our approach, called SPI with Baseline Bootstrapping (SPIBB), is inspired by the knows-what-it-knows paradigm: it bootstraps the trained policy with the baseline when the uncertainty is high. Our first algorithm, $Π_b$-SPIBB, comes with SPI theoretical guarantees. We also implement a variant, $Π_{\leq b}$-SPIBB, that is …
LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎电信学院 LTCI）；Microsoft Research Montréal（蒙特利尔微软研究院）
Batch Reinforcement Learning (Batch RL) consists in training a policy using trajectories collected with another policy, called the behavioural policy. Safe policy improvement (SPI) provides guarantees with high probability that the trained policy performs better than the behavioural policy, also called baseline in this setting. Previous work shows that the SPI objective improves mean performance as compared to using the basic RL objective, which boils down to solving the MDP with maximum likelihood. Here, we build on that work and improve more precisely the SPI with Baseline Bootstrapping algo…
DOF: Accelerating High-order Differential Operators with Forward Propagation
National Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）
PolyMicros: Bootstrapping a Foundation Model for Polycrystalline Material Structure
School of Computational Science and Engineering（计算科学与工程学院）；Georgia Institute of Technology（佐治亚理工学院）；Center for Integrated Nanotechnology（集成纳米技术中心）；Sandia National Laboratories（沙伊达国家实验室）；George W. Woodruff School of Mechanical Engineering（乔治·W·伍德鲁夫机械工程学院）
Deep reinforcement learning (DRL) provides a new way to generate robot control policy. However, the process of training control policy requires lengthy exploration, resulting in a low sample efficiency of reinforcement learning (RL) in real-world tasks. Both imitation learning (IL) and learning from demonstrations (LfD) improve the training process by using expert demonstrations, but imperfect expert demonstrations can mislead policy improvement. Offline to Online reinforcement learning requires a lot of offline data to initialize the policy, and distribution shift can easily lead to performan…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Using predictive models to identify patterns that can act as biomarkers for different neuropathoglogical conditions is becoming highly prevalent. In this paper, we consider the problem of Autism Spectrum Disorder (ASD) classification where previous work has shown that it can be beneficial to incorporate a wide variety of meta features, such as socio-cultural traits, into predictive modeling. A graph-based approach naturally suits these scenarios, where a contextual graph captures traits that characterize a population, while the specific brain activity patterns are utilized as a multivariate si…
By introducing several improvements to the AlphaZero process and architecture, we greatly accelerate self-play learning in Go, achieving a 50x reduction in computation over comparable methods. Like AlphaZero and replications such as ELF OpenGo and Leela Zero, our bot KataGo only learns from neural-net-guided Monte Carlo tree search self-play. But whereas AlphaZero required thousands of TPUs over several days and ELF required thousands of GPUs over two weeks, KataGo surpasses ELF's final model after only 19 days on fewer than 30 GPUs. Much of the speedup involves non-domain-specific improvement…
Nanjing University of Posts and Telecommunications（南京邮电大学）；Nanjing Normal University（南京师范大学）
Graph neural networks (GNNs) have been widely applied to numerous fields. A recent work which combines layered structure and residual connection proposes an improved deep architecture to extend CAmouflage-REsistant GNN (CARE-GNN) to deep models named as Residual Layered CARE-GNN (RLC-GNN), which forms self-correcting and incremental learning mechanism, and achieves significant performance improvements on fraud detection task. However, we spot three issues of RLC-GNN, which are the usage of neighboring information reaching limitation, the training difficulty which is inherent problem to deep mo…
AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play
Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）；Rutgers University（罗格斯大学）；SUNY Albany（纽约州立大学阿尔巴尼分校）；UT Southwestern Medical Center（德克萨斯西南医学中心）
This paper presents methods to accelerate recurrent neural network based language models (RNNLMs) for online speech recognition systems. Firstly, a lossy compression of the past hidden layer outputs (history vector) with caching is introduced in order to reduce the number of LM queries. Next, RNNLM computations are deployed in a CPU-GPU hybrid manner, which computes each layer of the model on a more advantageous platform. The added overhead by data exchanges between CPU and GPU is compensated through a frame-wise batching strategy. The performance of the proposed methods evaluated on LibriSpee…
Accelerating Proximal Policy Optimization Learning Using Task Prediction for Solving Environments with Delayed Rewards
Boston University（波士顿大学）；Worcester Polytechnic Institute（伍斯特理工学院）；MIT Lincoln Laboratory（麻省理工学院林肯实验室）；Anduril Industries（安杜里尔工业公司）；Lehigh University（利哈伊大学）；University of Maryland（马里兰大学）
