Multi-dimensional functional data arises in numerous modern scientific experimental and observational studies. In this paper we focus on longitudinal functional data, a structured form of multidimensional functional data. Operating within a longitudinal functional framework we aim to capture low dimensional interpretable features. We propose a computationally efficient nonparametric Bayesian method to simultaneously smooth observed data, estimate conditional functional means and functional covariance surfaces. Statistical inference is based on Monte Carlo samples from the posterior measure through adaptive blocked Gibbs sampling. Several operative characteristics associated with the proposed modeling framework are assessed comparatively in a simulated environment. We illustrate the application of our work in two case studies. The first case study involves age-specific fertility collected over time for various countries. The second case study is an implicit learning experiment in children with Autism Spectrum Disorder (ASD).
It is reasonable to consider, in many cases, that individuals' latent traits have a hierarchical structure such that more general traits are a suitable composition of more specific ones. Existing item response models that account for such hierarchical structure feature have considerable limitations in terms of modelling and/or inference. Motivated by those limitations and the importance of the theme, this paper aims at proposing an improved methodology in terms of both modelling and inference to deal with hierarchically structured latent traits in an item response theory context. From a modell…
In recent years, conditional copulas, that allow dependence between variables to vary according to the values of one or more covariates, have attracted increasing attention. In high dimension, vine copulas offer greater flexibility compared to multivariate copulas, since they are constructed using bivariate copulas as building blocks. In this paper we present a novel inferential approach for multivariate distributions, which combines the flexibility of vine constructions with the advantages of Bayesian nonparametrics, not requiring the specification of parametric families for each pair copula.…
We present a nonparametric method for estimating the value and several derivatives of an unknown, sufficiently smooth real-valued function of real-valued arguments from a finite sample of points, where both the function arguments and the corresponding values are known only up to measurement errors having some assumed distribution and correlation structure. The method, Moving Taylor Bayesian Regression (MOTABAR), uses Bayesian updating to find the posterior mean of the coefficients of a Taylor polynomial of the function at a moving position of interest. When measurement errors are neglected, MO…
We introduce a highly efficient fully Bayesian approach for anisotropic multidimensional smoothing. The main challenge in this context is the Markov chain Monte Carlo update of the smoothing parameters as their full conditional posterior comprises a pseudo-determinant that appears to be intractable at first sight. As a consequence, most existing implementations are computationally feasible only for the estimation of two-dimensional tensor product smooths, which is, however, too restrictive for many applications. In this paper, we break this barrier and derive closed-form expressions for the lo…
A Bayesian Approach Toward Robust Multidimensional Ellipsoid-Specific Fitting
Hong Kong Institute of Science & Innovation Chinese Academy of Sciences（中国科学院香港创新研究院）；Academy of Mathematics and Systems Science, CAS（中国科学院数学与系统科学研究院）；BAAI（北京智源人工智能研究院）；Peking University（北京大学）；School of Computer Science and Technology, Shandong University（山东大学计算机科学与技术学院）；Institute of Automation, CAS（中国科学院自动化研究所）
Given a sample from a discretely observed multidimensional compound Poisson process, we study the problem of nonparametric estimation of its jump size density $r_0$ and intensity $λ_0$. We take a nonparametric Bayesian approach to the problem and determine posterior contraction rates in this context, which, under some assumptions, we argue to be optimal posterior contraction rates. In particular, our results imply the existence of Bayesian point estimates that converge to the true parameter pair $(r_0,λ_0)$ at these rates. To the best of our knowledge, construction of nonparametric density est…
Differential equations (DEs) are commonly used to describe dynamic systems evolving in one (ordinary differential equations or ODEs) or in more than one dimensions (partial differential equations or PDEs). In real data applications the parameters involved in the DE models are usually unknown and need to be estimated from the available measurements together with the state function. In this paper, we present frequentist and Bayesian approaches for the joint estimation of the parameters and of the state functions involved in PDEs. We also propose two strategies to include differential (initial an…
Hierarchical learning models, such as mixture models and Bayesian networks, are widely employed for unsupervised learning tasks, such as clustering analysis. They consist of observable and hidden variables, which represent the given data and their hidden generation process, respectively. It has been pointed out that conventional statistical analysis is not applicable to these models, because redundancy of the latent variable produces singularities in the parameter space. In recent years, a method based on algebraic geometry has allowed us to analyze the accuracy of predicting observable variab…
Social network data are relational data recorded among a group of actors, interacting in different contexts. Often, the same set of actors can be characterized by multiple social relations, captured by a multidimensional network. A common situation is that of colleagues working in the same institution, whose social interactions can be defined on professional and personal levels. In addition, individuals in a network tend to interact more frequently with similar others, naturally creating communities. Latent space models for network data are useful to recover clustering of the actors, as they a…
Big Bayes is the computationally intensive co-application of big data and large, expressive Bayesian models for the analysis of complex phenomena in scientific inference and statistical learning. Standing as an example, Bayesian multidimensional scaling (MDS) can help scientists learn viral trajectories through space-time, but its computational burden prevents its wider use. Crucial MDS model calculations scale quadratically in the number of observations. We partially mitigate this limitation through massive parallelization using multi-core central processing units, instruction-level vectoriza…
Multivariate functional data arise in a wide range of applications. One fundamental task is to understand the causal relationships among these functional objects of interest, which has not yet been fully explored. In this article, we develop a novel Bayesian network model for multivariate functional data where the conditional independence and causal structure are both encoded by a directed acyclic graph. Specifically, we allow the functional objects to deviate from Gaussian process, which is adopted by most existing functional data analysis models. The more reasonable non-Gaussian assumption i…
Typical Bayesian inference requires parameter identification via likelihood parameterization, which has invited criticism for being less flexible than the Frequentist framework and subject to misspecification. Though misspecification may be avoided by functional parameter inference under a nonparametric model space, there does not exist a flexible Bayesian semiparametric model that would allow full control over the marginal prior over any general functional parameter. We present the technique of $θ$-augmentation which helps us manipulate nonparametric models into semiparametric ones that direc…
We propose a way to construct fiducial distributions for a multidimensional parameter using a step-by-step conditional procedure related to the inferential importance of the components of the parameter. For discrete models, in which the non-uniqueness of the fiducial distribution is well known, we propose to use the geometric mean of the "extreme cases" and show its good behavior with respect to the more traditional arithmetic mean. Connections with the generalized fiducial inference approach developed by Hannig and with confidence distributions are also analyzed. The suggested procedure stron…
We develop a Bayesian graphical modeling framework for functional data for correlated multivariate random variables observed over a continuous domain. Our method leads to graphical Markov models for functional data which allows the graphs to vary over the functional domain. The model involves estimation of graphical models that evolve functionally in a nonparametric fashion while accounting for within-functional correlations and borrowing strength across functional positions so contiguous locations are encouraged but not forced to have similar graph structure and edge strength. We utilize a st…
Functional data are defined as realizations of random functions (mostly smooth functions) varying over a continuum, which are usually collected with measurement errors on discretized grids. In order to accurately smooth noisy functional observations and deal with the issue of high-dimensional observation grids, we propose a novel Bayesian method based on the Bayesian hierarchical model with a Gaussian-Wishart process prior and basis function representations. We first derive an induced model for the basis-function coefficients of the functional data, and then use this model to conduct posterior…
We propose a Bayesian approach to estimating parameters in multiclass functional models. Unordered multinomial probit, ordered multinomial probit and multinomial logistic models are considered. We use finite random series priors based on a suitable basis such as B-splines in these three multinomial models, and classify the functional data using the Bayes rule. We average over models based on the marginal likelihood estimated from Markov Chain Monte Carlo (MCMC) output. Posterior contraction rates for the three multinomial models are computed. We also consider Bayesian linear and quadratic disc…
We propose a new Bayesian Markov switching regression model for multidimensional arrays (tensors) of binary time series. We assume a zero-inflated logit regression with time-varying parameters and apply it to multilayer temporal networks. The original contribution is threefold. First, to avoid over-fitting we propose a parsimonious parametrization based on a low-rank decomposition of the tensor of regression coefficients. Second, we assume the parameters are driven by a hidden Markov chain, thus allowing for structural changes in the network topology. We follow a Bayesian approach to inference…
We present a Bayesian approach for modeling multivariate, dependent functional data. To account for the three dominant structural features in the data--functional, time dependent, and multivariate components--we extend hierarchical dynamic linear models for multivariate time series to the functional data setting. We also develop Bayesian spline theory in a more general constrained optimization framework. The proposed methods identify a time-invariant functional basis for the functional observations, which is smooth and interpretable, and can be made common across multivariate observations for …
The present work describes simulation studies to compare the performances of bayesian wavelet shrinkage methods in estimating component curves from aggregated functional data. To do so, five methods were considered: the bayesian shrinkage rule under logistic prior by Sousa (2020), bayesian shrinkage rule under beta prior by Sousa et al. (2020), Large Posterior Mode method by Cutillo et al. (2008), Amplitude-scale invariant Bayes Estimator by Figueiredo and Nowak (2001) and Bayesian Adaptive Multiresolution Smoother by Vidakovic and Ruggeri (2001). Further, the so called Donoho-Johnstone test f…
We consider a Bayesian functional data analysis for observations measured as extremely long sequences. Splitting the sequence into a number of small windows with manageable length, the windows may not be independent especially when they are neighboring to each other. We propose to utilize Bayesian smoothing splines to estimate individual functional patterns within each window and to establish transition models for parameters involved in each window to address the dependent structure between windows. The functional difference of groups of individuals at each window can be evaluated by Bayes Fac…
Historical Functional Linear Models (HFLM) quantify associations between a functional predictor and functional outcome where the predictor is an exposure variable that occurs before, or at least concurrently with, the outcome. Current work on the HFLM is largely limited to frequentist estimation techniques that employ spline-based basis representations. In this work, we propose a novel use of the discrete wavelet-packet transformation, which has not previously been used in functional models, to estimate historical relationships in a fully Bayesian model. Since inference has not been an emphasi…
The spatial lag model (SLM) has been widely studied in the literature for spatialised data modeling in various disciplines such as geography, economics, demography, regional sciences, etc. This is an extension of the classical linear model that takes into account the proximity of spatial units in modeling. The extension of the SLM model in the functional framework (the FSLM model) as well as its estimation by the truncated maximum likelihood technique have been proposed by \cite{Ahmed}. In this paper, we propose a Bayesian estimation of the FSLM model. The Bayesian MCMC technique is used as es…
The extraordinary advancements in neuroscientific technology for brain recordings over the last decades have led to increasingly complex spatio-temporal datasets. To reduce oversimplifications, new models have been developed to be able to identify meaningful patterns and new insights within a highly demanding data environment. To this extent, we propose a new model called parameter clustering functional Principal Component Analysis (PCl-fPCA) that merges ideas from Functional Data Analysis and Bayesian nonparametrics to obtain a flexible and computationally feasible signal reconstruction and e…
Modeling with multidimensional arrays, or tensors, often presents a problem due to high dimensionality. In addition, these structures typically exhibit inherent sparsity, requiring the use of regularization methods to properly characterize an association between a tensor covariate and a scalar response. We propose a Bayesian method to efficiently model a scalar response with a tensor covariate using the Tucker tensor decomposition in order to retain the spatial relationship within a tensor coefficient, while reducing the number of parameters varying within the model and applying regularization…
The functional linear regression model is a common tool to determine the relationship between a scalar outcome and a functional predictor seen as a function of time. This paper focuses on the Bayesian estimation of the support of the coefficient function. To this aim we propose a parsimonious and adaptive decomposition of the coefficient function as a step function, and a model including a prior distribution that we name Bayesian functional Linear regression with Sparse Step functions (Bliss). The aim of the method is to recover areas of time which influences the most the outcome. A Bayes esti…
We propose a Bayesian method to detect change points for functional data. We extract the features of a sequence of functional data by the discrete wavelet transform (DWT), and treat each sequence of feature independently. We believe there is potentially a change in each feature at possibly different time points. The functional data evolves through such changes throughout the sequences of observations. The change point for this sequence of functional data is the cumulative effect of changes in all features. We assign the features with priors which incorporate the characteristic of the wavelet c…
We propose a multidimensional tensor clustering approach for studying how professional basketball players' shooting patterns vary over court locations and game time. Unlike most existing methods that only study continuous-valued tensors or have to assume the same cluster structure along different tensor directions, we propose a Bayesian nonparametric model that deals with count-valued tensors and projects the heterogeneity among players onto tensor dimensions while allowing cluster structures to be different over directions. Our method is fully probabilistic; hence allows simultaneous inferenc…
U. Cambridge（剑桥大学）；WUSTL（圣路易斯华盛顿大学）；Columbia U.（哥伦比亚大学）
Gaussian processes are typically used for smoothing and interpolation on small datasets. We introduce a new Bayesian nonparametric framework -- GPatt -- enabling automatic pattern extrapolation with Gaussian processes on large multidimensional datasets. GPatt unifies and extends highly expressive kernels and fast exact inference techniques. Without human intervention -- no hand crafting of kernel features, and no sophisticated initialisation procedures -- we show that GPatt can solve large scale pattern extrapolation, inpainting, and kernel discovery problems, including a problem with 383400 t…
In this paper we present a family of algorithms that can simultaneously align and cluster sets of multidimensional curves measured on a discrete time grid. Our approach is based on a generative mixture model that allows non-linear time warping of the observed curves relative to the mean curves within the clusters. We also allow for arbitrary discrete-valued translation of the time axis, random real-valued offsets of the measured curves, and additive measurement noise. The resulting model can be viewed as a dynamic Bayesian network with a special transition structure that allows effective infer…
Functional data are often extremely high-dimensional and exhibit strong dependence structures but can often prove valuable for both prediction and inference. The literature on functional data analysis is well developed; however, there has been very little work involving functional data in complex survey settings. Motivated by physical activity monitor data from the National Health and Nutrition Examination Survey (NHANES), we develop a Bayesian model for functional covariates that can properly account for the survey design. Our approach is intended for non-Gaussian data and can be applied in m…
In recent years, neuroimaging has undergone a paradigm shift, moving away from the traditional brain mapping approach toward developing integrated, multivariate brain models that can predict categories of mental events. However, large interindividual differences in brain anatomy and functional localization after standard anatomical alignment remain a major limitation in performing this analysis, as it leads to feature misalignment across subjects in subsequent predictive models.
We investigate the issue of bandwidth estimation in a nonparametric functional regression model with function-valued, continuous real-valued and discrete-valued regressors under the framework of unknown error density. Extending from the recent work of Shang (2013, Computational Statistics & Data Analysis), we approximate the unknown error density by a kernel density estimator of residuals, where the regression function is estimated by the functional Nadaraya-Watson estimator that admits mixed types of regressors. We derive a kernel likelihood and posterior density for the bandwidth parameters …
Purdue University（普渡大学）；IBM T. J. Watson Research Center（IBM T·J·沃森研究中心）
Infinite Tucker Decomposition (InfTucker) and random function prior models, as nonparametric Bayesian models on infinite exchangeable arrays, are more powerful models than widely-used multilinear factorization methods including Tucker and PARAFAC decomposition, (partly) due to their capability of modeling nonlinear relationships between array elements. Despite their great predictive performance and sound theoretical foundations, they cannot handle massive data due to a prohibitively high training time. To overcome this limitation, we present Distributed Infinite Tucker (DINTUCKER), a large-sca…
This study examines the optimal selections of bandwidth and semi-metric for a functional partial linear model. Our proposed method begins by estimating the unknown error density using a kernel density estimator of residuals, where the regression function, consisting of parametric and nonparametric components, can be estimated by functional principal component and functional Nadayara-Watson estimators. The estimation accuracy of the regression function and error density crucially depends on the optimal estimations of bandwidth and semi-metric. A Bayesian method is utilized to simultaneously est…
Functional BART with Shape Priors: A Bayesian Tree Approach to Constrained Functional Regression
The University of Texas Health Science Center at Houston（德克萨斯大学健康科学中心休斯顿分校）；Beijing Technology and Business University（北京工商大学）；Beijing Normal-Hong Kong Baptist University（香港中文大学（深圳））
Bayesian geoacoustic inversion using mixture density network
Research Institute of Petroleum Exploration & Development PetroChina（中国石油勘探开发研究院）；College of Meteorology and Oceanography National University of Defense Technology（国防科技大学气象海洋学院）
We consider non-parametric Bayesian estimation of the drift coefficient of a one-dimensional stochastic differential equation from discrete-time observations on the solution of this equation. Under suitable regularity conditions that are weaker than those previosly suggested in the literature, we establish posterior consistency in this context. Furthermore, we show that posterior consistency extends to the multidimensional setting as well, which, to the best of our knowledge, is a new result in this setting.
Graphical models, used to express conditional dependence between random variables observed at various nodes, are used extensively in many fields such as genetics, neuroscience, and social network analysis. While most current statistical methods for estimating graphical models focus on scalar data, there is interest in estimating analogous dependence structures when the data observed at each node are functional, such as signals or images. In this paper, we propose a fully Bayesian regularization scheme for estimating functional graphical models. We first consider a direct Bayesian analog of the…
