Deep generative models rely on their inductive bias to facilitate generalization, especially for problems with high dimensional data, like images. However, empirical studies have shown that variational autoencoders (VAE) and generative adversarial networks (GAN) lack the generalization ability that occurs naturally in human perception. For example, humans can visualize a woman smiling after only seeing a smiling man. On the contrary, the standard conditional VAE (cVAE) is unable to generate unseen attribute combinations. To this end, we extend cVAE by introducing a multilinear latent conditioning framework that captures the multiplicative interactions between the attributes. We implement two variants of our model and demonstrate their efficacy on MNIST, Fashion-MNIST and CelebA. Altogether, we design a novel conditioning framework that can be used with any architecture to synthesize unseen attribute combinations.
Shortest-Path Flow Matching with Mixture-Conditioned Bases for OOD Generalization to Unseen Conditions
Wellcome Sanger Institute（Wellcome桑格研究所）；Institute for Molecular Medicine Finland (FIMM)（芬兰分子医学研究所）；University of Helsinki（赫尔辛基大学）；School of Computing and Information Systems（计算与信息系统学院）；University of Cambridge（剑桥大学）；Cambridge Centre for AI in Medicine（剑桥人工智能医学中心）；MediaTek Research（MediaTek研究）；Cambridge Stem Cell Institute（剑桥干细胞研究所）
We propose NeRF-VAE, a 3D scene generative model that incorporates geometric structure via NeRF and differentiable volume rendering. In contrast to NeRF, our model takes into account shared structure across scenes, and is able to infer the structure of a novel scene -- without the need to re-train -- using amortized inference. NeRF-VAE's explicit 3D rendering process further contrasts previous generative models with convolution-based rendering which lacks geometric structure. Our model is a VAE that learns a distribution over radiance fields by conditioning them on a latent scene representatio…
ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation
Institute for AI, University of Stuttgart（斯图加特大学人工智能研究所）；IMPRS-IS（国际马克斯·普朗克智能系统研究学院）；Institute of Theoretical Chemistry, University of Stuttgart（斯图加特大学理论化学研究所）
Stellenbosch University（斯泰伦博斯大学）；The University of Edinburgh（爱丁堡大学）
For our submission to the ZeroSpeech 2019 challenge, we apply discrete latent-variable neural networks to unlabelled speech and use the discovered units for speech synthesis. Unsupervised discrete subword modelling could be useful for studies of phonetic category learning in infants or in low-resource speech technology requiring symbolic input. We use an autoencoder (AE) architecture with intermediate discretisation. We decouple acoustic unit discovery from speaker modelling by conditioning the AE's decoder on the training speaker identity. At test time, unit discovery is performed on speech f…
Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Department of Diagnostic Imaging, Royal Free London NHS Foundation Trust（伦敦皇家自由NHS基金会信托诊断影像科）
Sonography synthesis has a wide range of applications, including medical procedure simulation, clinical training and multimodality image registration. In this paper, we propose a machine learning approach to simulate ultrasound images at given 3D spatial locations (relative to the patient anatomy), based on conditional generative adversarial networks (GANs). In particular, we introduce a novel neural network architecture that can sample anatomically accurate images conditionally on spatial position of the (real or mock) freehand ultrasound probe. To ensure an effective and efficient spatial in…
Parameter-Efficient Conditioning for Material Generalization in Graph-Based Simulators
Department of Civil, Architecture and Environmental Engineering（土木、建筑与环境工程系）；The Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究所）
Center for Machine Learning (ZML)（机器学习中心（ZML））；Institute for Flow in Additively Manufactured Porous Structures (ISAPS)（添加剂制造多孔结构流动研究所（ISAPS））；Heilbronn University of Applied Sciences（海德堡应用科学大学）
Sculpting Quantum Landscapes: Fubini-Study Metric Conditioning for Geometry Aware Learning in Parameterized Quantum Circuits
Quantum Physics and Spintronic Team, LPMC, Faculty of Sciences Ben M’sick, Hassan II University of Casablanca, Morocco（量子物理与自旋电子团队，LPMC，科学学院本·梅西克，卡萨布兰卡哈桑二世大学，摩洛哥）
