In this paper, we present a density estimation framework based on tree tensor-network states. The proposed method consists of determining the tree topology with Chow-Liu algorithm, and obtaining a linear system of equations that defines the tensor-network components via sketching techniques. Novel choices of sketch functions are developed in order to consider graphical models that contain loops. Sample complexity guarantees are provided and further corroborated by numerical experiments.
The Central Limit Theorem (CLT) for extrinsic and intrinsic means on manifolds is extended to a generalization of Fréchet means. Examples are the Procrustes mean for 3D Kendall shapes as well as a mean introduced by Ziezold. This allows for one-sample tests previously not possible, and to numerically assess the `inconsistency of the Procrustes mean' for a perturbation model and `inconsistency' within a model recently proposed for diffusion tensor imaging. Also it is shown that the CLT can be extended to mildly rank deficient diffusion tensors. An application to forestry gives the temporal evol…
Dipartimento di Informatica - Università di Pisa（比萨大学信息学系）
The paper introduces two new aggregation functions to encode structural knowledge from tree-structured data. They leverage the Canonical and Tensor-Train decompositions to yield expressive context aggregation while limiting the number of model parameters. Finally, we define two novel neural recursive models for trees leveraging such aggregation functions, and we test them on two tree classification tasks, showing the advantage of proposed models when tree outdegree increases.
Department of Statistics, Rice University（里士大学统计系）；Computational Research Division, Lawrence Berkeley National Laboratory（伯克利国家实验室计算研究部）；Department of Statistics, University of California Davis（加州大学戴维斯分校统计系）；Department of Statistics, University of Chicago（芝加哥大学统计系）
Institute of Physics, Chinese Academy of Sciences（中国科学院物理研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Theoretical Physics, Chinese Academy of Sciences（中国科学院理论物理研究所）
Matrix product states (MPS), a tensor network designed for one-dimensional quantum systems, has been recently proposed for generative modeling of natural data (such as images) in terms of `Born machine'. However, the exponential decay of correlation in MPS restricts its representation power heavily for modeling complex data such as natural images. In this work, we push forward the effort of applying tensor networks to machine learning by employing the Tree Tensor Network (TTN) which exhibits balanced performance in expressibility and efficient training and sampling. We design the tree tensor n…
Università degli Studi di Genova（热那亚大学）；IE University（IE大学）；Universitat de València（瓦伦西亚大学）
Generative models for classification use the joint probability distribution of the class variable and the features to construct a decision rule. Among generative models, Bayesian networks and naive Bayes classifiers are the most commonly used and provide a clear graphical representation of the relationship among all variables. However, these have the disadvantage of highly restricting the type of relationships that could exist, by not allowing for context-specific independences. Here we introduce a new class of generative classifiers, called staged tree classifiers, which formally account for …
Bottom-Up Hidden Tree Markov Model is a highly expressive model for tree-structured data. Unfortunately, it cannot be used in practice due to the intractable size of its state-transition matrix. We propose a new approximation which lies on the Tucker factorisation of tensors. The probabilistic interpretation of such approximation allows us to define a new probabilistic model for tree-structured data. Hence, we define the new approximated model and we derive its learning algorithm. Then, we empirically assess the effective power of the new model evaluating it on two different tasks. In both cas…
Insilico Medicine（英矽智能）；National Research University Higher School of Economics（俄罗斯国立高等经济研究大学）
Generative models produce realistic objects in many domains, including text, image, video, and audio synthesis. Most popular models---Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs)---usually employ a standard Gaussian distribution as a prior. Previous works show that the richer family of prior distributions may help to avoid the mode collapse problem in GANs and to improve the evidence lower bound in VAEs. We propose a new family of prior distributions---Tensor Ring Induced Prior (TRIP)---that packs an exponential number of Gaussians into a high-dimensional lattice …
Generative Learning of Continuous Data by Tensor Networks
Zapata AI（扎帕塔人工智能公司）；Perimeter Institute for Theoretical Physics（圆周理论物理研究所）；Institute for Quantum Computing, University of Waterloo（滑铁卢大学量子计算研究所）；DIRO, Université de Montréal（蒙特利尔大学计算机科学与运筹学系）；CIFAR AI（加拿大高级研究所人工智能项目）
Perimeter Institute for Theoretical Physics（圆周理论物理研究所）；Alphabet (Google) X（Alphabet（谷歌）X）
TensorNetwork is an open source library for implementing tensor network algorithms in TensorFlow. We describe a tree tensor network (TTN) algorithm for approximating the ground state of either a periodic quantum spin chain (1D) or a lattice model on a thin torus (2D), and implement the algorithm using TensorNetwork. We use a standard energy minimization procedure over a TTN ansatz with bond dimension $χ$, with a computational cost that scales as $O(χ^4)$. Using bond dimension $χ\in [32,256]$ we compare the use of CPUs with GPUs and observe significant computational speed-ups, up to a factor of…
University of Chinese Academy of Sciences（中国科学院大学）；Tianjin Polytechnic University（天津工业大学）；Capital Normal University（首都师范大学）
Tensor network (TN) has recently triggered extensive interests in developing machine-learning models in quantum many-body Hilbert space. Here we purpose a generative TN classification (GTNC) approach for supervised learning. The strategy is to train the generative TN for each class of the samples to construct the classifiers. The classification is implemented by comparing the distance in the many-body Hilbert space. The numerical experiments by GTNC show impressive performance on the MNIST and Fashion-MNIST dataset. The testing accuracy is competitive to the state-of-the-art convolutional neur…
Generative Modeling via Hierarchical Tensor Sketching
Committee on Computational and Applied Mathematics, University of Chicago（计算与应用数学委员会，芝加哥大学）；Center for Computational Quantum Physics, Flatiron Institute（计算量子物理中心，Flatiron研究所）；Department of Statistics, University of Chicago（统计学系，芝加哥大学）
While many recent advances in deep reinforcement learning (RL) rely on model-free methods, model-based approaches remain an alluring prospect for their potential to exploit unsupervised data to learn environment model. In this work, we provide an extensive study on the design of deep generative models for RL environments and propose a sample efficient and robust method to learn the model of Atari environments. We deploy this model and propose generative adversarial tree search (GATS) a deep RL algorithm that learns the environment model and implements Monte Carlo tree search (MCTS) on the lear…
Centrale Nantes（南特中央理工学院）；Laboratoire de Mathématiques Jean Leray（让·勒雷数学实验室）；CNRS（法国国家科学研究中心）；Naval Group（海军集团）；Airbus Central Research & Technology（空客中央研究与技术中心）
We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density with respect to a product measure, possibly discrete for handling the case of discrete random variables. After discussing the representation of classical model classes in tree-based tensor formats, we present learning algorithms based on empirical risk minimization using a $L^2$ …
We present a generative approach to classify scarcely observed longitudinal patient trajectories. The available time series are represented as tensors and factorized using generative deep recurrent neural networks. The learned factors represent the patient data in a compact way and can then be used in a downstream classification task. For more robustness and accuracy in the predictions, we used an ensemble of those deep generative models to mimic Bayesian posterior sampling. We illustrate the performance of our architecture on an intensive-care case study of in-hospital mortality prediction wi…
Discovering the latent structure from many observed variables is an important yet challenging learning task. Existing approaches for discovering latent structures often require the unknown number of hidden states as an input. In this paper, we propose a quartet based approach which is \emph{agnostic} to this number. The key contribution is a novel rank characterization of the tensor associated with the marginal distribution of a quartet. This characterization allows us to design a \emph{nuclear norm} based test for resolving quartet relations. We then use the quartet test as a subroutine in a …
Tree tensor networks, or tree-based tensor formats, are prominent model classes for the approximation of high-dimensional functions in computational and data science. They correspond to sum-product neural networks with a sparse connectivity associated with a dimension tree and widths given by a tuple of tensor ranks. The approximation power of these models has been proved to be (near to) optimal for classical smoothness classes. However, in an empirical risk minimization framework with a limited number of observations, the dimension tree and ranks should be selected carefully to balance estima…
PhyloVAE: Unsupervised Learning of Phylogenetic Trees via Variational Autoencoders
School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Center for Statistical Science, Peking University（北京大学统计科学中心）；Computational Biology Program, Fred Hutchinson Cancer Research Center（弗雷德·哈钦森癌症研究中心计算生物学项目）；Howard Hughes Medical Institute（霍华德·休斯医学研究所）
Casting neural networks in generative frameworks is a highly sought-after endeavor these days. Contemporary methods, such as Generative Adversarial Networks, capture some of the generative capabilities, but not all. In particular, they lack the ability of tractable marginalization, and thus are not suitable for many tasks. Other methods, based on arithmetic circuits and sum-product networks, do allow tractable marginalization, but their performance is challenged by the need to learn the structure of a circuit. Building on the tractability of arithmetic circuits, we leverage concepts from tenso…
The resemblance between the methods used in quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and flexibilities. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multi-scale entanglement renormalization ansatz. This approach introduc…
