We propose a multivariate generative model to capture the complex dependence structure often encountered in business and financial data. Our model features heterogeneous and asymmetric tail dependence between all pairs of individual dimensions while also allowing heterogeneity and asymmetry in the tails of the marginals. A significant merit of our model structure is that it is not prone to error propagation in the parameter estimation process, hence very scalable, as the dimensions of datasets grow large. However, the likelihood methods are infeasible for parameter estimation in our case due to the lack of a closed-form density function. Instead, we devise a novel moment learning algorithm to learn the parameters. To demonstrate the effectiveness of the model and its estimator, we test them on simulated as well as real-world datasets. Results show that this framework gives better finite-sample performance compared to the copula-based benchmarks as well as recent similar models.
University of Cambridge（剑桥大学）；University of Vienna（维也纳大学）；Microsoft Research Cambridge（微软剑桥研究院）
Deep generative models often perform poorly in real-world applications due to the heterogeneity of natural data sets. Heterogeneity arises from data containing different types of features (categorical, ordinal, continuous, etc.) and features of the same type having different marginal distributions. We propose an extension of variational autoencoders (VAEs) called VAEM to handle such heterogeneous data. VAEM is a deep generative model that is trained in a two stage manner such that the first stage provides a more uniform representation of the data to the second stage, thereby sidestepping the p…
Duke University（杜克大学）；Infinia ML, Inc（Infinia ML公司）；Microsoft Dynamics 365 AI Research（微软Dynamics 365人工智能研究院）；Facebook, Inc（脸书公司）
We propose a novel graph-driven generative model, that unifies multiple heterogeneous learning tasks into the same framework. The proposed model is based on the fact that heterogeneous learning tasks, which correspond to different generative processes, often rely on data with a shared graph structure. Accordingly, our model combines a graph convolutional network (GCN) with multiple variational autoencoders, thus embedding the nodes of the graph i.e., samples for the tasks) in a uniform manner while specializing their organization and usage to different tasks. With a focus on healthcare applica…
The commonly used latent space embedding techniques, such as Principal Component Analysis, Factor Analysis, and manifold learning techniques, are typically used for learning effective representations of homogeneous data. However, they do not readily extend to heterogeneous data that are a combination of numerical and categorical variables, e.g., arising from linked GPS and text data. In this paper, we are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. The learned generative model provides latent unified representation…
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology（大数据知识工程重点实验室（教育部））
Tsinghua University（清华大学）；AHI Fintech（安恒信息金融科技）；Youku Cognitive and Intelligent Lab, Alibaba Group（阿里巴巴集团优酷认知与智能实验室）
Since the label collecting is prohibitive and time-consuming, unsupervised methods are preferred in applications such as fraud detection. Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions. Existing methods propose to model the data clusters on selected dimensions, yet globally omitting any dimension may damage the pattern of certain clusters. To address the above issues, we propose a novel unsupervised generative…
Game-theoretic distributed learning of generative models for heterogeneous data collections
Faculty of Computer Science, Dresden University of Technology（计算机科学系，德累斯顿技术大学）；Department of Cybernetics, Czech Technical University in Prague（cybernetics系，捷克技术大学布拉格分校）
Generative Adversarial Networks (GAN) are a powerful methodology and can be used for unsupervised anomaly detection, where current techniques have limitations such as the accurate detection of anomalies near the tail of a distribution. GANs generally do not guarantee the existence of a probability density and are susceptible to mode collapse, while few GANs use likelihood to reduce mode collapse. In this paper, we create a GAN-based tail formation model for anomaly detection, the Tail of distribution GAN (TailGAN), to generate samples on the tail of the data distribution and detect anomalies n…
Accurate and automated detection of anomalous samples in a natural image dataset can be accomplished with a probabilistic model for end-to-end modeling of images. Such images have heterogeneous complexity, however, and a probabilistic model overlooks simply shaped objects with small anomalies. This is because the probabilistic model assigns undesirably lower likelihoods to complexly shaped objects that are nevertheless consistent with set standards. To overcome this difficulty, we propose an unregularized score for deep generative models (DGMs), which are generative models leveraging deep neur…
Massachusetts Institute of Technology, CSAIL（麻省理工学院计算机科学与人工智能实验室）
Generative Adversarial Networks (GANs) are popular tools for generative modeling. The dynamics of their adversarial learning give rise to convergence pathologies during training such as mode and discriminator collapse. In machine learning, ensembles of predictors demonstrate better results than a single predictor for many tasks. In this study, we apply two evolutionary algorithms (EAs) to create ensembles to re-purpose generative models, i.e., given a set of heterogeneous generators that were optimized for one objective (e.g., minimize Frechet Inception Distance), create ensembles of them for …
Normalizing flows, a popular class of deep generative models, often fail to represent extreme phenomena observed in real-world processes. In particular, existing normalizing flow architectures struggle to model multivariate extremes, characterized by heavy-tailed marginal distributions and asymmetric tail dependence among variables. In light of this shortcoming, we propose COMET (COpula Multivariate ExTreme) Flows, which decompose the process of modeling a joint distribution into two parts: (i) modeling its marginal distributions, and (ii) modeling its copula distribution. COMET Flows capture …
Flow Battery Manifold Design with Heterogeneous Inputs Through Generative Adversarial Neural Networks
Department of Electrical and Computer Engineering, Duke University（电气与计算机工程系，杜克大学）；School of Chemistry and Chemical Engineering, Queen’s University Belfast（化学与化学工程学院，女王大学贝尔法斯特分校）；School of Mechanical and Materials Engineering, University College Dublin（机械与材料工程学院，都柏林大学学院）
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机学院与工程学院）；Taiyuan University of Technology（太原理工大学）
Generative models (GMs) such as Generative Adversary Network (GAN) and Variational Auto-Encoder (VAE) have thrived these years and achieved high quality results in generating new samples. Especially in Computer Vision, GMs have been used in image inpainting, denoising and completion, which can be treated as the inference from observed pixels to corrupted pixels. However, images are hierarchically structured which are quite different from many real-world inference scenarios with non-hierarchical features. These inference scenarios contain heterogeneous stochastic variables and irregular mutual …
University of Notre Dame（圣母大学）；Brandeis University（布兰迪斯大学）
Generative self-supervised learning (SSL), especially masked autoencoders, has become one of the most exciting learning paradigms and has shown great potential in handling graph data. However, real-world graphs are always heterogeneous, which poses three critical challenges that existing methods ignore: 1) how to capture complex graph structure? 2) how to incorporate various node attributes? and 3) how to encode different node positions? In light of this, we study the problem of generative SSL on heterogeneous graphs and propose HGMAE, a novel heterogeneous graph masked autoencoder model to ad…
Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Models
School of Computing Technologies（计算技术学院）；RMIT University（皇家墨尔本理工大学）；Department of Computing Technologies（计算技术系）；Swinburne University of Technology（斯威本理工大学）
Generative AI-Powered Plugin for Robust Federated Learning in Heterogeneous IoT Networks
Institute of Information & Communications Technology Planning & Evaluation (IITP)-ITRC (Information Technology Research Center)（信息与通信技术规划与评估院（IITP）-ITRC（信息技术研究中心））
Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts
National Engineering Research Center for Big Data Technology；System Lab, Cluster；Grid Computing Lab, School of Computer Science；Technology, Huazhong University of Science；DIRO, Université de Montréal \& Mila \& Canada CIFAR AI Chair, Canada
Wasserstein Bounds for generative diffusion models with Gaussian tail targets
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore（数学科学系，物理与数学科学学院，南洋理工大学，新加坡）
TU Kaiserslautern（凯泽斯劳滕工业大学）；Fraunhofer ITWM（弗劳恩霍夫工业数学研究所）
Deep generative networks such as GANs and normalizing flows flourish in the context of high-dimensional tasks such as image generation. However, so far exact modeling or extrapolation of distributional properties such as the tail asymptotics generated by a generative network is not available. In this paper, we address this issue for the first time in the deep learning literature by making two novel contributions. First, we derive upper bounds for the tails that can be expressed by a generative network and demonstrate Lp-space related properties. There we show specifically that in various situa…
Ritsumeikan University（立命馆大学）；Osaka University（大阪大学）；The University of Electro-Communications（电气通信大学）；The University of Tokyo（东京大学）
This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-to-tail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKE…
Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices
University of Macau（澳门大学）；Zhejiang University of Technology（浙江工业大学）；Guangdong University of Technology（广东工业大学）；Nanyang Technological University（南洋理工大学）；University of Waterloo（滑铁卢大学）
Bielefeld University（比勒费尔德大学）；Queensland University of Technology（昆士兰科技大学）
We investigate a reinforcement approach for distributed sensing based on the latent space derived from multi-modal deep generative models. Our contribution provides insights to the following benefits: Detections can be exchanged effectively between robots equipped with uni-modal sensors due to a shared latent representation of information that is trained by a Variational Auto Encoder (VAE). Sensor-fusion can be applied asynchronously due to the generative feature of the VAE. Deep Q-Networks (DQNs) are trained to minimize uncertainty in latent space by coordinating robots to a Point-of-Interest…
Cornell University（康奈尔大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Hawai’i at Manoa（夏威夷大学马诺阿分校）
This work is the first to employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) towards learning a forward and an inverse solution operator of partial differential equations (PDEs). Even though the proposed framework could be applied as a surrogate model for the solution of any PDEs, here we focus on steady-state solutions of coupled hydro-mechanical processes in heterogeneous porous media. Strongly heterogeneous material properties, which translate to the heterogeneity of coefficients of the PDEs and discontinuous features in the s…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；Renmin University of China（中国人民大学）
Structure-based drug design is drawing growing attentions in computer-aided drug discovery. Compared with the virtual screening approach where a pre-defined library of compounds are computationally screened, de novo drug design based on the structure of a target protein can provide novel drug candidates. In this paper, we present a generative solution named TamGent (Target-aware molecule generator with Transformer) that can directly generate candidate drugs from scratch for a given target, overcoming the limits imposed by existing compound libraries. Following the Transformer framework (a stat…
Modeling biological soft tissue is complex in part due to material heterogeneity. Microstructural patterns, which play a major role in defining the mechanical behavior of these tissues, are both challenging to characterize, and difficult to simulate. Recently, machine learning-based methods to predict the mechanical behavior of heterogeneous materials have made it possible to more thoroughly explore the massive input parameter space associated with heterogeneous blocks of material. Specifically, we can train machine learning (ML) models to closely approximate computationally expensive heteroge…
MarketGANs: Multivariate financial time-series data augmentation using generative adversarial networks
Department of Mathematics, Sungkyunkwan University, Republic of Korea（淑阳大学数学系）；Department of Financial Engineering, Ajou University, Republic of Korea（东亚大学金融工程系）；Department of FinTech, SKK Business School, Sungkyunkwan University, Republic of Korea（淑阳大学SKK商学院金融科技系）
Chongqing University of Posts and Telecommunications（重庆邮电大学）；University of Guelph（圭尔夫大学）；Hosei University（法政大学）
Emerging six generation (6G) is the integration of heterogeneous wireless networks, which can seamlessly support anywhere and anytime networking. But high Quality-of-Trust should be offered by 6G to meet mobile user expectations. Artificial intelligence (AI) is considered as one of the most important components in 6G. Then AI-based trust management is a promising paradigm to provide trusted and reliable services. In this article, a generative adversarial learning-enabled trust management method is presented for 6G wireless networks. Some typical AI-based trust management schemes are first revi…
CTSyn: A Foundation Model for Cross Tabular Data Generation
Department of Statistics and Data Science, University of California Los Angeles（统计与数据科学系，加州大学洛杉矶分校）；Department of Computer Science, University of California Los Angeles（计算机科学系，加州大学洛杉矶分校）
Quantum-Enhanced Generative Models for Rare Event Prediction
Department of Software Engineering(ÉTS), Université du Québec（魁北克大学软件工程系）；Department Of Computational Sciences,The University of Faisalabad（费萨利班大学计算科学系）；Department of Computer Science, SZABIST University（SZABIST大学计算机科学系）
