Hyperbolic Neural Networks (HNNs), operating in hyperbolic space, have been widely applied in recent years, motivated by the existence of an optimal embedding in hyperbolic space that can preserve data hierarchical relationships (termed Hierarchical Representation Capability, HRC) more accurately than Euclidean space. However, there is no evidence to suggest that HNNs can achieve this theoretical optimal embedding, leading to much research being built on flawed motivations. In this paper, we propose a benchmark for evaluating HRC and conduct a comprehensive analysis of why HNNs are effective through large-scale experiments. Inspired by the analysis results, we propose several pre-training strategies to enhance HRC and improve the performance of downstream tasks, further validating the reliability of the analysis. Experiments show that HNNs cannot achieve the theoretical optimal embedding. The HRC is significantly affected by the optimization objectives and hierarchical structures, and enhancing HRC through pre-training strategies can significantly improve the performance of HNNs.
We present a large scale hyperbolic recommender system. We discuss why hyperbolic geometry is a more suitable underlying geometry for many recommendation systems and cover the fundamental milestones and insights that we have gained from its development. In doing so, we demonstrate the viability of hyperbolic geometry for recommender systems, showing that they significantly outperform Euclidean models on datasets with the properties of complex networks. Key to the success of our approach are the novel choice of underlying hyperbolic model and the use of the Einstein midpoint to define an asymme…
University of Stuttgart（斯图加特大学）；University of Bonn（波恩大学）；Monash University（莫纳什大学）；University of Southampton（南安普顿大学）
Recent knowledge graph (KG) embeddings have been advanced by hyperbolic geometry due to its superior capability for representing hierarchies. The topological structures of real-world KGs, however, are rather heterogeneous, i.e., a KG is composed of multiple distinct hierarchies and non-hierarchical graph structures. Therefore, a homogeneous (either Euclidean or hyperbolic) geometry is not sufficient for fairly representing such heterogeneous structures. To capture the topological heterogeneity of KGs, we present an ultrahyperbolic KG embedding (UltraE) in an ultrahyperbolic (or pseudo-Riemanni…
Microsoft Research（微软研究院）；IBM Research（IBM研究院）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）
This paper focuses on the problem of unsupervised alignment of hierarchical data such as ontologies or lexical databases. This is a problem that appears across areas, from natural language processing to bioinformatics, and is typically solved by appeal to outside knowledge bases and label-textual similarity. In contrast, we approach the problem from a purely geometric perspective: given only a vector-space representation of the items in the two hierarchies, we seek to infer correspondences across them. Our work derives from and interweaves hyperbolic-space representations for hierarchical data…
Beijing Institute of Technology（北京理工大学）；Guangdong Laboratory of Machine Perception and Intelligent Computing（广东省机器感知与智能计算实验室）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）
South China Normal University（华南师范大学）；Research Institute of China Telecom Corporate Ltd.（中国电信股份有限公司研究院）
Introducing prior auxiliary information from the knowledge graph (KG) to assist the user-item graph can improve the comprehensive performance of the recommender system. Many recent studies show that the ensemble properties of hyperbolic spaces fit the scale-free and hierarchical characteristics exhibited in the above two types of graphs well. However, existing hyperbolic methods ignore the consideration of equivariance, thus they cannot generalize symmetric features under given transformations, which seriously limits the capability of the model. Moreover, they cannot balance preserving the het…
Hyperbolic Graph Neural Networks Under the Microscope: The Role of Geometry-Task Alignment
University of Rome Tor Vergata（罗马大学Tor Vergata）；University of Basel（巴塞尔大学）；University of Rome Tor Vergata Harvard Medical School（罗马大学Tor Vergata哈佛医学院）；Bocconi University Bocconi Institute for Data Science and Analytics（博科尼大学博科尼数据科学与分析研究所）；UiT The Arctic University of Norway（UiT北极大学）
Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect t…
ASIL: Augmented Structural Information Learning for Deep Graph Clustering in Hyperbolic Space
Beijing University of Posts and Telecommunications（北京邮电大学）；North China Electric Power University（华北电力大学）；DiDi Chuxing（滴滴出行）；Beihang University（北航）；University of Illinois at Chicago（伊利诺伊大学香槟分校）
Naval Postgraduate School（海军研究生院）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
As demonstrated in many areas of real-life applications, neural networks have the capability of dealing with high dimensional data. In the fields of optimal control and dynamical systems, the same capability was studied and verified in many published results in recent years. Towards the goal of revealing the underlying reason why neural networks are capable of solving some high dimensional problems, we develop an algebraic framework and an approximation theory for compositional functions and their neural network approximations. The theoretical foundation is developed in a way so that it suppor…
