This paper is concerned with general nonlinear regression models where the predictor variables are subject to Berkson-type measurement errors. The measurement errors are assumed to have a general parametric distribution, which is not necessarily normal. In addition, the distribution of the random error in the regression equation is nonparametric. A minimum distance estimator is proposed, which is based on the first two conditional moments of the response variable given the observed predictor variables. To overcome the possible computational difficulty of minimizing an objective function which involves multiple integrals, a simulation-based estimator is constructed. Consistency and asymptotic normality for both estimators are derived under fairly general regularity conditions.
Given a sample $\{X_i\}_{i=1}^n$ from $f_X$, we construct kernel density estimators for $f_Y$, the convolution of $f_X$ with a known error density $f_ε$. This problem is known as density estimation with Berkson error and has applications in epidemiology and astronomy. Little is understood about bandwidth selection for Berkson density estimation. We compare three approaches to selecting the bandwidth both asymptotically, using large sample approximations to the MISE, and at finite samples, using simulations. Our results highlight the relationship between the structure of the error $f_ε$ and the…
This paper establishes that so-called instrumental variables enable the identification and the estimation of a fully nonparametric regression model with Berkson-type measurement error in the regressors. An estimator is proposed and proven to be consistent. Its practical performance and feasibility are investigated via Monte Carlo simulations as well as through an epidemiological application investigating the effect of particulate air pollution on respiratory health. These examples illustrate that Berkson errors can clearly not be neglected in nonlinear regression models and that the proposed m…
The present paper considers a problem of estimating a linear functional $Φ=\int_{-\infty}^\infty φ(x) f(x)dx$ of an unknown deconvolution density $f$ on the basis of i.i.d. observations $Y_i = θ_i + ξ_i$ where $ξ_i$ has a known pdf $g$ and $f$ is the pdf of $θ_i$. Although various aspects and particular cases of this problem have been treated by a number of authors, there are still many gaps. In particular, there are no minimax lower bounds for an estimator of $Φ$ for an arbitrary function $φ$. The general upper risk bounds cover only the case when the Fourier transform of $φ$ exists. Moreover…
This paper considers a nonlinear quantile model with change-points. The quantile estimation method, which as a particular case includes median model, is more robust with respect to other traditional methods when model errors contain outliers. Under relatively weak assumptions, the convergence rate and asymptotic distribution of change-point and of regression parameter estimators are obtained. Numerical study by Monte Carlo simulations shows the performance of the proposed method for nonlinear model with change-points.
We consider the problem of threshold estimation for autoregressive time series with a "space switching" in the situation, when the regression is nonlinear and the innovations have a smooth, possibly non Gaussian, probability density. Assuming that the unknown threshold parameter is sampled from a continuous positive density, we find the asymptotic distribution of the Bayes estimator. As usually in the singular estimation problems, the sequence of Bayes estimators is found to be asymptotically efficient, attaining the minimax risk lower bound.
We consider a re-sampling scheme for estimation of the population parameters in the mixed effects nonlinear regression models of the type use for example in clinical pharmacokinetics, say. We provide an estimation procedure which {\it recycles}, via random weighting, the relevant two-stage parameters estimates to construct consistent estimates of the sampling distribution of the various estimates. We establish the asymptotic consistency and asymptotic normality of the resampled estimates and demonstrate the applicability of the {\it recycling} approach in a small simulation study and via examp…
We study the maximum likehood estimator and least squares estimator for drift parameters of nonlinear reflected stochastic differential equations based on continuous observations. Under some regular conditions, we obtain the consistency and give the asymptotic distributions of the two estimators. We briefly remark that our methods could be applied the the reflected stochastic processes with only one-sided reflecting barrier spontaneously. Numerical studies show that the proposed estimators are adequate for practical use.
In this paper, we study parametric nonlinear regression under the Harris recurrent Markov chain framework. We first consider the nonlinear least squares estimators of the parameters in the homoskedastic case, and establish asymptotic theory for the proposed estimators. Our results show that the convergence rates for the estimators rely not only on the properties of the nonlinear regression function, but also on the number of regenerations for the Harris recurrent Markov chain. Furthermore, we discuss the estimation of the parameter vector in a conditional volatility function, and apply our res…
We investigate a semiparametric regression model where one gets noisy non linear non invertible functions of the observations. We focus on the application to bearings-only tracking. We first investigate the least squares estimator and prove its consistency and asymptotic normality under mild assumptions. We study the semiparametric likelihood process and prove local asymptotic normality of the model. This allows to define the efficient Fisher information as a lower bound for the asymptotic variance of regular estimators, and to prove that the parametric likelihood estimator is regular and asym…
This paper studies a nonlinear filtering problem over an infinite time interval. The signal to be estimated is driven by a stochastic partial differential equation involves unknown parameters. Based on discrete observation, strongly consistent estimators of the parameters are derived at first. With the optimal filter given by Bayes formula, the uniqueness of invariant measure for the signal-filter pair has been verified. The paper then establishes approximation to the optimal filter, showing that the pathwise average distance, per unit time, of the computed approximating filter from the optima…
We consider a resampling scheme for parameters estimates in nonlinear regression models. We provide an estimation procedure which recycles, via random weighting, the relevant parameters estimates to construct consistent estimates of the sampling distribution of the various estimates. We establish the asymptotic normality of the resampled estimates and demonstrate the applicability of the recycling approach in a small simulation study and via example.
We study the estimation of $β$ for the nonlinear model $y = f(X\sp{\top}β) + ε$ when $f$ is a nonlinear transformation that is known, $β$ has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters ($n\ll p$). We show that in order to bound the $L_2$ error of the $L_0$ regularized estimator $\hatβ$, i.e., $\|\hatβ- β\|_2$, it is sufficient to establish two conditions. Based on this, we obtain bounds of the $L_2$ error for (1) $L_0$ regularized maximum likelihood estimation (MLE) for exponential linear models and (2) $L_0$ regularized least square …
A function based nonlinear least squares estimation (FNLSE) method is proposed and investigated in parameter estimation of Jelinski-Moranda software reliability model. FNLSE extends the potential fitting functions of traditional least squares estimation (LSE), and takes the logarithm transformed nonlinear least squares estimation (LogLSE) as a special case. A novel power transformation function based nonlinear least squares estimation (powLSE) is proposed and applied to the parameter estimation of Jelinski-Moranda model. Solved with Newton-Raphson method, Both LogLSE and powLSE of Jelinski-Mor…
Future University Hakodate（函馆未来大学）；National Graduate Institute for Policy Studies（政策研究大学院大学）；RIKEN AIP（理化学研究所先进智能项目中心）
Contrastive learning is a recent promising approach in unsupervised representation learning where a feature representation of data is learned by solving a pseudo classification problem from unlabelled data. However, it is not straightforward to understand what representation contrastive learning yields. In addition, contrastive learning is often based on the maximum likelihood estimation, which tends to be vulnerable to the contamination by outliers. To promote the understanding to contrastive learning, this paper first theoretically shows a connection to maximization of mutual information (MI…
This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished with a number of convergence results, including a central limit theorem with an asymptotic variance that can be shown to be uniformly bounded in time. Using the proposed filter derivative estimator we design a recursive maximum likelihood algorithm updating the parameters according the gradient of the one-step predictor log-likelihood. The efficiency of this …
Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian. We estimate the gradient and Hessian using Fisher's identity in combination with a smoothing algorithm. We explore two approximations of the log-likelihood and of the solution of the smoothing problem. The first is a linearization approximation which is computationally cheap, but the accuracy typically varies between models. The second is a sampling approximation which is asymptotically …
We propose a new method for the Maximum Likelihood Estimator (MLE) of nonlinear mixed effects models when the variance matrix of Gaussian random effects has a prescribed pattern of zeros (PPZ). The method consists in coupling the recently developed Iterative Conditional Fitting (ICF) algorithm with the Expectation Maximization (EM) algorithm. It provides positive definite estimates for any sample size, and does not rely on any structural assumption on the PPZ. It can be easily adapted to many versions of EM.
Technische Universität Dortmund（多特蒙德工业大学）；Slovak University of Technology in Bratislava（斯洛伐克理工大学）
We study approaches to robust model-based design of experiments in the context of maximum-likelihood estimation. These approaches provide robustification of model-based methodologies for the design of optimal experiments by accounting for the effect of the parametric uncertainty. We study the problem of robust optimal design of experiments in the framework of nonlinear least-squares parameter estimation using linearized confidence regions. We investigate several well-known robustification frameworks in this respect and propose a novel methodology based on multi-stage robust optimization. The p…
Princeton University（普林斯顿大学）；Israel Institute of Technology（以色列理工学院）；Rutgers University（罗格斯大学）
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top β^* ) + ε$, where $f$ is nonlinear. To recover $β^*$, we propose an $\ell_1$-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of $f$. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. In addition, we provide an efficient algorithm tha…
We study two nonlinear methods for statistical linear inverse problems when the operator is not known. The two constructions combine Galerkin regularization and wavelet thresholding. Their performances depend on the underlying structure of the operator, quantified by an index of sparsity. We prove their rate-optimality and adaptivity properties over Besov classes.
Parameter estimation in a class of heteroscedastic time series models is investigated. The existence of conditional least-squares and conditional likelihood estimators is proved. Their consistency and their asymptotic normality are established. Kernel estimators of the noise's density and its derivatives are defined and shown to be uniformly consistent. A simulation experiment conducted shows that the estimators perform well for large sample size.
We study the adaptive minimax estimation of non-linear integral functionals of a density and extend the results obtained for linear and quadratic functionals to general functionals. The typical rate optimal non-adaptive minimax estimators of "smooth" non-linear functionals are higher order U-statistics. Since Lepski's method requires tight control of tails of such estimators, we bypass such calculations by a modification of Lepski's method which is applicable in such situations. As a necessary ingredient, we also provide a method to control higher order moments of minimax estimator of cubic in…
In a recent work (arXiv:0910.2517), for nonlinear models with sparse underlying linear structures, we studied the error bounds of $\ell_0$-regularized estimation. In this note, we show that $\ell_1$-regularized estimation in some important cases can achieve the same order of error bounds as those in the aforementioned work.
Our objective is to estimate the unknown compositional input from its output response through an unknown system after estimating the inverse of the original system with a training set. The proposed methods using artificial neural networks (ANNs) can compete with the optimal bounds for linear systems, where convex optimization theory applies, and demonstrate promising results for nonlinear system inversions. We performed extensive experiments by designing numerous different types of nonlinear systems.
