We study latent geometry as an explicit component of representation quality in data-scarce learning. For an encoder (ϕ), we define (Q_{β,γ}(ϕ)=I(ϕ(X);Y)-β\mathcal C(ϕ)-γd_{\mathrm{int}}(ϕ)), combining task-relevant information with penalties for curvature and intrinsic latent dimension. Thus geometry becomes part of the bottleneck criterion, not only a post hoc diagnostic.
Under smooth-manifold, loss-transfer, and estimator-concentration assumptions, we derive non-asymptotic low-label generalization bounds where intrinsic dimension and covering complexity enter explicitly. We characterize the information--geometry frontier and prove empirical-surrogate consistency. The analysis links encoder geometry to learning through latent covering numbers, loss-class entropy, and uniform deviation.
We instantiate the theory as \texttt{V-GIB}, adding curvature and dimension penalties to variational bottleneck training. Real low-label benchmarks compare \texttt{V-GIB} with ERM, VIB, and ablations across (1%)--(20%) label fractions. Results show improved performance and reduced geometric complexity in several regimes, especially FashionMNIST and CIFAR-10, while confirming that no fixed regularizer is universally dominant.
Multi-Station WiFi CSI Sensing Framework Robust to Station-wise Feature Missingness and Limited Labeled Data
School of Engineering, Institute of Science Tokyo（东京科学研究院工程学院）；Wireless Systems R&D Department, Infrastructure Systems R&D Center, Corporate Laboratory, Toshiba Corporation（东芝公司无线系统研发部、基础设施系统研发中心、企业实验室）
Harvard John A Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院，哈佛大学）；University of Basel（巴塞尔大学）
Estimating the causal effects of an intervention from high-dimensional observational data is difficult due to the presence of confounding. The task is often complicated by the fact that we may have a systematic missingness in our data at test time. Our approach uses the information bottleneck to perform a low-dimensional compression of covariates by explicitly considering the relevance of information. Based on the sufficiently reduced covariate, we transfer the relevant information to cases where data is missing at test time, allowing us to reliably and accurately estimate the effects of an in…
University of California, Los Angeles（加州大学洛杉矶分校）；University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
Integration of data from multiple omics techniques is becoming increasingly important in biomedical research. Due to non-uniformity and technical limitations in omics platforms, such integrative analyses on multiple omics, which we refer to as views, involve learning from incomplete observations with various view-missing patterns. This is challenging because i) complex interactions within and across observed views need to be properly addressed for optimal predictive power and ii) observations with various view-missing patterns need to be flexibly integrated. To address such challenges, we prop…
Counterfactual learning for dealing with missing-not-at-random data (MNAR) is an intriguing topic in the recommendation literature since MNAR data are ubiquitous in modern recommender systems. Missing-at-random (MAR) data, namely randomized controlled trials (RCTs), are usually required by most previous counterfactual learning methods for debiasing learning. However, the execution of RCTs is extraordinarily expensive in practice. To circumvent the use of RCTs, we build an information-theoretic counterfactual variational information bottleneck (CVIB), as an alternative for debiasing learning wi…
Multimodal Federated Learning With Missing Modalities through Feature Imputation Network
University of Aberdeen（阿伯丁大学）；NepAl Applied Mathematics and Informatics Institute for research（尼泊尔应用数学与信息技术研究所）；West Virginia University（西弗吉尼亚大学）；Fogsphere (Redev.AI)（Fogsphere（Redev.AI））
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）
The existence of external (``side'') semantic knowledge has been shown to result in more expressive computational event models. To enable the use of side information that may be noisy or missing, we propose a semi-supervised information bottleneck-based discrete latent variable model. We reparameterize the model's discrete variables with auxiliary continuous latent variables and a light-weight hierarchical structure. Our model is learned to minimize the mutual information between the observed data and optional side knowledge that is not already captured by the new, auxiliary variables. We theo…
CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning
School of Electronics and Information Technology, Sun Yat-Sen University（中山大学电子与信息学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Computer Science, South China Normal University（华南师范大学计算机科学学院）；Desay SV Automotive Co., Ltd（德赛股份有限公司）；Pazhou Laboratory（琶洲实验室）
University of Chinese Academy of Sciences（中国科学院大学）；Search Lab（360搜索实验室）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
There are threefold challenges in emotion recognition. First, it is difficult to recognize human's emotional states only considering a single modality. Second, it is expensive to manually annotate the emotional data. Third, emotional data often suffers from missing modalities due to unforeseeable sensor malfunction or configuration issues. In this paper, we address all these problems under a novel multi-view deep generative framework. Specifically, we propose to model the statistical relationships of multi-modality emotional data using multiple modality-specific generative networks with a shar…
Gradient Guided Hypotheses: A unified solution to enable machine learning models on scarce and noisy data regimes
In-Silico Discovery (ISD), Janssen Research & Development, Janssen Pharmaceutica N.V（杨森制药公司研发部计算机模拟发现）；ISIC, EPFL, Laboratory of Artificial Chemical Intelligence (LIAC)（洛桑联邦理工学院化学与化学工程学院分子与生物化学研究所人工智能化学实验室）；National Centre of Competence in Research (NCCR) Catalysis, Ecole Polytechnique Fédérale de Lausanne (EPFL)（瑞士国家科研能力中心催化研究，洛桑联邦理工学院）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；NEC Laboratories America（美国NEC实验室）
Graph neural network (GNN), as a powerful representation learning model on graph data, attracts much attention across various disciplines. However, recent studies show that GNN is vulnerable to adversarial attacks. How to make GNN more robust? What are the key vulnerabilities in GNN? How to address the vulnerabilities and defense GNN against the adversarial attacks? In this paper, we propose DefNet, an effective adversarial defense framework for GNNs. In particular, we first investigate the latent vulnerabilities in every layer of GNNs and propose corresponding strategies including dual-stage …
Scarcity of labeled data is a bottleneck for supervised learning models. A paradigm that has evolved for dealing with this problem is data programming. An existing data programming paradigm allows human supervision to be provided as a set of discrete labeling functions (LF) that output possibly noisy labels to input instances and a generative modelfor consolidating the weak labels. We enhance and generalize this paradigm by supporting functions that output a continuous score (instead of a hard label) that noisily correlates with labels. We show across five applications that continuous LFs are …
