Nonlinear mixed effects models have become a standard platform for analysis when data is in the form of continuous and repeated measurements of subjects from a population of interest, while temporal profiles of subjects commonly follow a nonlinear tendency. While frequentist analysis of nonlinear mixed effects models has a long history, Bayesian analysis of the models has received comparatively little attention until the late 1980s due primarily to the time-consuming nature of Bayesian computation. Since the early 1990s Bayesian approaches for the models began to emerge to leverage rapid developments in computing power, and recently, have received significant attention due to (1) superiority to quantify the uncertainty of parameter estimation; (2) utility to incorporate prior knowledge into the models; and (3) flexibility to match exactly the increasing complexity of scientific research arising from diverse industrial and academic fields. This review article presents an overview of modeling strategies to implement Bayesian approaches for the nonlinear mixed effects models, ranging from designing a scientific question out of real-life problems to practical computations.
Many psychological theories can be operationalized as linear inequality constraints on the parameters of multinomial distributions (e.g., discrete choice analysis). These constraints can be described in two equivalent ways: Either as the solution set to a system of linear inequalities or as the convex hull of a set of extremal points (vertices). For both representations, we describe a general Gibbs sampler for drawing posterior samples in order to carry out Bayesian analyses. We also summarize alternative sampling methods for estimating Bayes factors for these model representations using the e…
This paper presents an overview of some of the concepts of Bayesian Learning. The number of scientific and industrial applications of Bayesian learning has been growing in size rapidly over the last few decades. This process has started with the wide use of Markov Chain Monte Carlo methods that emerged as a dominant computational technique for Bayesian in the early 1990's. Since then Bayesian learning has spread well across several fields from robotics and machine learning to medical applications. This paper provides an overview of some of the widely used concepts and shows several application…
Is Repeated Bayesian Interim Analysis Consequence-Free?
The University of Texas MD Anderson Cancer Center（德克萨斯大学安德森癌症中心）；Louisiana State University（路易斯安那州立大学）；Office of Biostatistics, OTS/CDER, Food and Drug Administration（食品药品监督管理局生物统计办公室）
In this paper, I develop a formula for estimating Bayes factors directly from minimal summary statistics produced in repeated measures analysis of variance designs. The formula, which requires knowing only the $F$-statistic, the number of subjects, and the number of repeated measurements per subject, is based on the BIC approximation of the Bayes factor, a common default method for Bayesian computation with linear models. In addition to providing computational examples, I report a simulation study in which I demonstrate that the formula compares favorably to a recently developed, more complex …
With the arrival of the R packages nlme and lme4, linear mixed models (LMMs) have come to be widely used in experimentally-driven areas like psychology, linguistics, and cognitive science. This tutorial provides a practical introduction to fitting LMMs in a Bayesian framework using the probabilistic programming language Stan. We choose Stan (rather than WinBUGS or JAGS) because it provides an elegant and scalable framework for fitting models in most of the standard applications of LMMs. We ease the reader into fitting increasingly complex LMMs, first using a two-condition repeated measures sel…
Whilst most psychological research focuses on differences in means, a growing body of literature demonstrates the value of considering differences in intra-individual variability. Compared to the number of methods available for analyzing mean differences, there is a paucity of methods available for analyzing intra-individual variability, particularly when variability is treated as a predictor. In the present article, we first reviewed methods of analyzing intra-individual variability as an outcome, including the individual standard deviation (ISD) and some recent methods. We then introduced a …
I begin my discussion by giving an overview of the main results. Then I proceed to touch upon issues about whether the credible ball constructed can be interpreted as a confidence ball, suggestions on reducing computational costs, and posterior consistency or contraction rates.
In this paper the Bayesian analysis is applied to assign a probability density to the value of a quantity having a definite sign. This analysis is logically consistent with the results, positive or negative, of repeated measurements. Results are used to estimate the atom density shift in a caesium fountain clock. The comparison with the classical statistical analysis is also reported and the advantages of the Bayesian approach for the realization of the time unit are discussed.
Bayesian statistics has gained great momentum since the computational developments of the 1990s. Gradually, advances in Bayesian methodology and software have made Bayesian techniques much more accessible to applied statisticians and, in turn, have potentially transformed Bayesian education at the undergraduate level. This article provides an overview on the various options for implementing Bayesian computational methods motivated to achieve particular learning outcomes. The advantages and disadvantages of each computational method are described based on the authors' experience in using these …
Bayesian networks have been used as a mechanism to represent the joint distribution of multiple random variables in a flexible yet interpretable manner. One major challenge in learning the structure of a Bayesian network is how to model networks which include a mixture of continuous and discrete random variables, known as hybrid Bayesian networks. This paper overviews the literature on approaches to handle hybrid Bayesian networks. Typically one of two approaches is taken: either the data are considered to have a joint distribution which is designed for a mixture of discrete and continuous var…
Existing state-wide data bases on prosecutors' decisions about juvenile offenders are important, yet often un-explored resources for understanding changes in patterns of judicial decisions over time. We investigate the extent and nature of change in judicial behavior toward juveniles following the enactment of a new set of mandatory registration policies between 1992 and 1996 via analyzing the data on prosecutors' decisions of moving forward for youths repeatedly charged with sexual violence in South Carolina. To analyze this longitudinal binary data, we use a random effects logistic regressio…
This manuscript introduces an \proglang{R} package called \pkg{trialr} that implements a collection of clinical trial methods in \proglang{Stan} and \proglang{R}. In this article, we explore three methods in detail. The first is the continual reassessment method for conducting phase I dose-finding trials that seek a maximum tolerable dose. The second is EffTox, a dose-finding design that scrutinises doses by joint efficacy and toxicity outcomes. The third is the augmented binary method for modelling the probability of treatment success in phase II oncology trials with reference to repeated mea…
Calibrating hierarchical Bayesian domain inference for a proportion
Continuing Academic Specialist Center for Statistical Training and Consulting, Michigan State University（密歇根州立大学统计培训与咨询中心继续教育学术专员）；Institute for Social Research, University of Michigan（密歇根大学社会研究所）
Bayesian hierarchical models are increasingly popular for realistic modelling and analysis of complex data. This trend is accompanied by the need for flexible, general, and computationally efficient methods for model criticism and conflict detection. Usually, a Bayesian hierarchical model incorporates a grouping of the individual data points, for example individuals in repeated measurement data. In such cases, the following question arises: Are any of the groups "outliers", or in conflict with the remaining groups? Existing general approaches aiming to answer such questions tend to be extremel…
We propose a model for unbalanced longitudinal data, where the univariate margins can be selected arbitrarily and the dependence structure is described with the help of a D-vine copula. We show that our approach is an extremely flexible extension of the widely used linear mixed model if the correlation is homogeneous over the considered individuals. As an alternative to joint maximum-likelihood a sequential estimation approach for the D-vine copula is provided and validated in a simulation study. The model can handle missing values without being forced to discard data. Since conditional distri…
This Chapter, "Overview of Approximate Bayesian Computation", is to appear as the first chapter in the forthcoming Handbook of Approximate Bayesian Computation (2018). It details the main ideas and concepts behind ABC methods with many examples and illustrations.
Bayesian approaches for handling covariate measurement error are well established, and yet arguably are still relatively little used by researchers. For some this is likely due to unfamiliarity or disagreement with the Bayesian inferential paradigm. For others a contributory factor is the inability of standard statistical packages to perform such Bayesian analyses. In this paper we first give an overview of the Bayesian approach to handling covariate measurement error, and contrast it with regression calibration (RC), arguably the most commonly adopted approach. We then argue why the Bayesian …
In literature there are several studies on the performance of Bayesian network structure learning algorithms. The focus of these studies is almost always the heuristics the learning algorithms are based on, i.e. the maximisation algorithms (in score-based algorithms) or the techniques for learning the dependencies of each variable (in constraint-based algorithms). In this paper we investigate how the use of permutation tests instead of parametric ones affects the performance of Bayesian network structure learning from discrete data. Shrinkage tests are also covered to provide a broad overview …
Simulations often involve the use of model parameters which are unknown or uncertain. For this reason, simulation experiments are often repeated for multiple combinations of parameter values, often iterating through parameter values lying on a fixed grid. However, the use of a discrete grid places limits on the dimension of the parameter space and creates the potential to miss important parameter combinations which fall in the gaps between grid points. Here we draw parallels with strategies for numerical integration and describe a Markov chain Monte-Carlo strategy for exploring parameter value…
We review typical design problems encountered in the planning of observational studies and propose a unifying framework that allows us to use the same concepts and notation for different problems. In the framework, the design is defined as a probability measure in the space of observational processes that determine whether the value of a variable is observed for a specific unit at the given time. The optimal design is then defined, according to Bayesian decision theory, to be the one that maximizes the expected utility related to the design. We present examples on the use of the framework and …
Most computational approaches to Bayesian experimental design require making posterior calculations repeatedly for a large number of potential designs and/or simulated datasets. This can be expensive and prohibit scaling up these methods to models with many parameters, or designs with many unknowns to select. We introduce an efficient alternative approach without posterior calculations, based on optimising the expected trace of the Fisher information, as discussed by Walker (2016). We illustrate drawbacks of this approach, including lack of invariance to reparameterisation and encouraging desi…
We present a nonparametric Bayesian joint model for multivariate continuous and categorical variables, with the intention of developing a flexible engine for multiple imputation of missing values. The model fuses Dirichlet process mixtures of multinomial distributions for categorical variables with Dirichlet process mixtures of multivariate normal distributions for continuous variables. We incorporate dependence between the continuous and categorical variables by (i) modeling the means of the normal distributions as component-specific functions of the categorical variables and (ii) forming dis…
Adversarial risk analysis (ARA) is a relatively new area of research that informs decision-making when facing intelligent opponents and uncertain outcomes. It enables an analyst to express her Bayesian beliefs about an opponent's utilities, capabilities, probabilities and the type of strategic calculation that the opponent is using. Within that framework, the analyst then solves the problem from the perspective of the opponent while placing subjective probability distributions on all unknown quantities. This produces a distribution over the actions of the opponent that permits the analyst to m…
While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians. In this review, we highlight the importance of prior choices for Bayesian deep learning and present an overview of different priors that have been proposed for (deep) Gaussian processes, variational autoencoders, and Bayesian neural networks. We also outline different methods of learning priors for these models from data. We hope to motivate practitioners in Bayesian deep learning to think more c…
Concerning systematic effects, the recommendation given in the GUM is to correct for them, but unfortunately no detailed information is available, how to do this. This publication will show, how systematic measurement deviations can be handled correctly based on the Bayesian probability theory. After a short overview about useful methods and tools, like the product rule of probability theory, Bayes' theorem, the principle of maximum entropy, and the marginalisation equation, an outline of a method to handle systematic measurement deviations is introduced. Finally some simple examples of practi…
Department of Statistics（统计系）；INSPER；Chicago Booth（芝加哥布斯商学院）；University of Chicago（芝加哥大学）；Volgenau School of Engineering（沃尔格瑙工程学院）；George Mason University（乔治梅森大学）
University of Freiburg（弗莱堡大学）；Meta；Bosch Center for Artificial Intelligence（博世人工智能中心）
When hyperparameter optimization of a machine learning algorithm is repeated for multiple datasets it is possible to transfer knowledge to an optimization run on a new dataset. We develop a new hyperparameter-free ensemble model for Bayesian optimization that is a generalization of two existing transfer learning extensions to Bayesian optimization and establish a worst-case bound compared to vanilla Bayesian optimization. Using a large collection of hyperparameter optimization benchmark problems, we demonstrate that our contributions substantially reduce optimization time compared to standard …
We give an overview of the role of information theory in statistics, and particularly in biostatistics. We recall the basic quantities in information theory; entropy, cross-entropy, conditional entropy, mutual information and Kullback-Leibler risk. Then we examine the role of information theory in estimation theory, where the log-klikelihood can be identified as being an estimator of a cross-entropy. Then the basic quantities are extended to estimators, leading to criteria for estimator selection, such as Akaike criterion and its extensions. Finally we investigate the use of these concepts in …
Traditional statistical analysis requires that the analysis process and data are independent. By contrast, the new field of adaptive data analysis hopes to understand and provide algorithms and accuracy guarantees for research as it is commonly performed in practice, as an iterative process of interacting repeatedly with the same data set, such as repeated tests against a holdout set. Previous work has defined a model with a rather strong lower bound on sample complexity in terms of the number of queries, $n\sim\sqrt q$, arguing that adaptive data analysis is much harder than static data analy…
University of Western Australia（西澳大学）；Murdoch University（莫道克大学）；Monash University（莫纳什大学）
Modern deep learning methods constitute incredibly powerful tools to tackle a myriad of challenging problems. However, since deep learning methods operate as black boxes, the uncertainty associated with their predictions is often challenging to quantify. Bayesian statistics offer a formalism to understand and quantify the uncertainty associated with deep neural network predictions. This tutorial provides an overview of the relevant literature and a complete toolset to design, implement, train, use and evaluate Bayesian Neural Networks, i.e. Stochastic Artificial Neural Networks trained using B…
This chapter provides a overview of Bayesian inference, mostly emphasising that it is a universal method for summarising uncertainty and making estimates and predictions using probability statements conditional on observed data and an assumed model (Gelman 2008). The Bayesian perspective is thus applicable to all aspects of statistical inference, while being open to the incorporation of information items resulting from earlier experiments and from expert opinions. We provide here the basic elements of Bayesian analysis when considered for standard models, refering to Marin and Robert (2007) an…
Family planning has been characterized by highly different strategic programs in India, including method-specific contraceptive targets, coercive sterilization, and more recent target-free approaches. These major changes in family planning policies over time have motivated a considerable interest towards assessing the effectiveness of the different programs, while understanding which subsets of the population have not been properly addressed. Current studies consider specific aspects of the above policies, including, for example, the factors associated with the choice of alternative contracept…
