Despite increasing accessibility to function data, effective methods for flexibly estimating underlying functional trend are still scarce. We thereby develop functional version of trend filtering for estimating trend of functional data indexed by time or on general graph by extending the conventional trend filtering, a powerful nonparametric trend estimation technique, for scalar data. We formulate the new trend filtering by introducing penalty terms based on $L_2$-norm of the differences of adjacent trend functions. We develop an efficient iteration algorithm for optimizing the objective function obtained by orthonormal basis expansion. Furthermore, we introduce additional penalty terms to eliminate redundant basis functions, which leads to automatic adaptation of the number of basis functions. The tuning parameter in the proposed method is selected via cross validation. We demonstrate the proposed method through simulation studies and applications to real world datasets.
We develop a fully Bayesian hierarchical model for trend filtering, itself a new development in nonparametric, univariate regression. The framework more broadly applies to the generalized lasso, but focus is on Bayesian trend filtering. We compare two shrinkage priors, double exponential and generalized double Pareto. A simulation study, comparing Bayesian trend filtering to the original formulation and a number of other popular methods shows our method to improve estimation error while maintaining if not improving coverage probability. Two time series data sets demonstrate Bayesian trend filt…
We address the problem of estimating smoothly varying baseline trends in time series data. This problem arises in a wide range of fields, including chemistry, macroeconomics, and medicine; however, our study is motivated by the analysis of data from low cost air quality sensors. Our methods extend the quantile trend filtering framework to enable the estimation of multiple quantile trends simultaneously while ensuring that the quantiles do not cross. To handle the computational challenge posed by very long time series, we propose a parallelizable alternating direction method of moments (ADMM) a…
We propose a novel approach for density estimation called histogram trend filtering. Our estimator arises from looking at surrogate Poisson model for counts of observations in a partition of the support of the data. We begin by showing consistency for a variational estimator for this density estimation problem. We then study a discrete estimator that can be efficiently found via convex optimization. We show that the estimator enjoys strong statistical guarantees, yet is much more practical and computationally efficient than other estimators that enjoy similar guarantees. Finally, in our simula…
Carnegie Mellon University（卡内基梅隆大学）；University of California at San Diego（加利福尼亚大学圣迭戈分校）；Marianas Labs（马里亚纳实验室）
We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filtering exhibits a level of local adaptivity unmatched by the usual $\ell_2$-based graph smoothers. It is also defined by a convex minimization problem that is readily solved (e.g., by fast ADMM or Newton algorithms). We demonstrate the merits of graph trend filtering through examples…
We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute $k$th order discrete derivatives over the input points. Perhaps not surprisingly, trend filtering estimates appear to have the structure of $k$th degree spline functions, with adaptively chosen knot points (we say ``appear'' here as trend filtering estimates are not really functions over continuous domains, and are only defined over…
While many approaches have been proposed for discovering abrupt changes in piecewise constant signals, few methods are available to capture these changes in piecewise polynomial signals. In this paper, we propose a change point detection method, PRUTF, based on trend filtering. By providing a comprehensive dual solution path for trend filtering, PRUTF allows us to discover change points of the underlying signal for either a given value of the regularization parameter or a specific number of steps of the algorithm. We demonstrate that the dual solution path constitutes a Gaussian bridge process…
There are many research works and methods about change point detection in the literature. However, there are only a few that provide inference for such change points after being estimated. This work mainly focuses on a statistical analysis of change points estimated by the PRUTF algorithm, which incorporates trend filtering to determine change points in piecewise polynomial signals. This paper develops a methodology to perform statistical inference, such as computing p-values and constructing confidence intervals in the newly developed post-selection inference framework. Our work concerns both…
This paper reviews a class of univariate piecewise polynomial functions known as discrete splines, which share properties analogous to the better-known class of spline functions, but where continuity in derivatives is replaced by (a suitable notion of) continuity in divided differences. As it happens, discrete splines bear connections to a wide array of developments in applied mathematics and statistics, from divided differences and Newton interpolation (dating back to over 300 years ago) to trend filtering (from the last 15 years). We survey these connections, and contribute some new perspect…
Graduate School of Economics, The University of Tokyo（东京大学经济学研究科）；Center for Spatial Information Science, The University of Tokyo（东京大学空间信息科学中心）
Due to developments in instruments and computers, functional observations are increasingly popular. However, effective methodologies for flexibly estimating the underlying trends with valid uncertainty quantification for a sequence of functional data (e.g. functional time series) are still scarce. In this work, we develop a locally adaptive smoothing method, called functional horseshoe smoothing, by introducing a shrinkage prior to the general order of differences of functional variables. This allows us to capture abrupt changes by making the most of the shrinkage capability and also to assess…
Here, we address the problem of trend estimation for functional time series. Existing contributions either deal with detecting a functional trend or assuming a simple model. They consider neither the estimation of a general functional trend nor the analysis of functional time series with a functional trend component. Similarly to univariate time series, we propose an alternative methodology to analyze functional time series, taking into account a functional trend component. We propose to estimate the functional trend by using a tensor product surface that is easy to implement, to interpret, an…
This article deals with the problem of functional classification for L2-valued random covariates when some of the covariates may have missing or unobservable fragments. Here, it is allowed for both the training sample as well as the new unclassified observation to have missing fragments in their functional covariates. Furthermore, unlike most previous results in the literature, where covariate fragments are typically assumed to be missing completely at random, we do not impose any such assumptions here. Given the observed segments of the curves, we construct a kernel-type classifier which is q…
A general approach for Bayesian filtering of multi-object systems is studied, with particular emphasis on the model where each object generates observations independently of other objects. The approach is based on variational calculus applied to generating functionals, using the general version of Faa di Bruno's formula for Gateaux differentials. This result enables us to determine some general formulae for the updated generating functional after the application of a multi-object analogue of Bayes' rule.
This paper assesses when the validity of difference-in-differences depends on functional form. We provide a novel characterization: the parallel trends assumption holds under all strictly monotonic transformations of the outcome if and only if a stronger ``parallel trends''-type condition holds for the cumulative distribution function of untreated potential outcomes. This condition for parallel trends to be insensitive to functional form is satisfied if and essentially only if the population can be partitioned into a subgroup for which treatment is effectively randomly assigned and a remaining…
We study the dynamics of cause--specific mortality rates among countries by considering them as compositions of functions. We develop a novel framework for such data structure, with particular attention to functional PCA. The application of this method to a subset of the WHO mortality database reveals the main modes of variation of cause--specific rates over years for men and women and enables us to perform clustering in the projected subspace. The results give many insights of the ongoing trends, only partially explained by past literature, that the considered countries are undergoing. We are…
We put forward a simple new randomized missing data (RMD) approach to robust filtering of state-space models, motivated by the idea that the inclusion of only a small fraction of available highly precise measurements can still extract most of the attainable efficiency gains for filtering latent states, estimating model parameters, and producing out-of-sample forecasts. In our general RMD framework we develop two alternative implementations: endogenous (RMD-N) and exogenous (RMD-X) randomization of missing data. A degree of robustness to outliers and model misspecification is achieved by purpos…
Spatiotemporal dynamics is central to a wide range of applications from climatology, computer vision to neural sciences. From temporal observations taken on a high-dimensional vector of spatial locations, we seek to derive knowledge about such dynamics via data assimilation and modeling. It is assumed that the observed spatiotemporal data represent superimposed lower-rank smooth oscillations and movements from a generative dynamic system, mixed with higher-rank random noises. Separating the signals from noises is essential for us to visualize, model and understand these lower-rank dynamic syst…
