Image classification has been studied extensively, but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training. We present a set of methods for leveraging information about the semantic hierarchy embedded in class labels. We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier and empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance. Taking a step further in this direction, we model more explicitly the label-label and label-image interactions using order-preserving embeddings governed by both Euclidean and hyperbolic geometries, prevalent in natural language, and tailor them to hierarchical image classification and representation learning. We empirically validate all the models on the hierarchical ETHEC dataset.
Learning graph representations via low-dimensional embeddings that preserve relevant network properties is an important class of problems in machine learning. We here present a novel method to embed directed acyclic graphs. Following prior work, we first advocate for using hyperbolic spaces which provably model tree-like structures better than Euclidean geometry. Second, we view hierarchical relations as partial orders defined using a family of nested geodesically convex cones. We prove that these entailment cones admit an optimal shape with a closed form expression both in the Euclidean and h…
Hyperbolic Multimodal Representation Learning for Biological Taxonomies
Simon Fraser University（西蒙弗雷泽大学）；University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）；University of Guelph（圭尔夫大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔机器智能研究所）；Aalborg University（奥胡斯大学）
HYDEN: Hyperbolic Density Representations for Medical Images and Reports
Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging（北京联合智能影像研究院智能诊断研究所）；School of Physics, Peking University（北京大学物理学院）
Vision and Language Pretraining has become the prevalent approach for tackling multimodal downstream tasks. The current trend is to move towards ever larger models and pretraining datasets. This computational headlong rush does not seem reasonable in the long term to move toward sustainable solutions, and de facto excludes academic laboratories with limited resources. In this work, we propose a new framework, dubbed ViCHA, that efficiently exploits the input data to boost the learning by: (a) a new hierarchical cross-modal alignment loss, (b) new self-supervised scheme based on masked image mo…
