自由度:搜索成本与自一致性
Degrees of Freedom: Search Cost and Self-consistency
AI总结:
本文提出修正搜索自由度以统一原始与增广空间的搜索成本,并引入名义与实际自由度的自一致性概念,为MARS等复杂程序提供校正方法,提升拟合性能。
摘要:
模型自由度($\df$)是统计学中的一个基本概念,因为它量化了拟合程序的灵活性,并且在模型选择中不可或缺。为了研究$\df$与拟合程序中自变量数量之间的差距,\textcite{tibshiraniDegreesFreedomModel2015}引入了\emph{搜索自由度}($\sdf$)概念,以解释模型选择过程中的搜索成本。然而,该定义存在两个局限性:它未考虑增广空间中的拟合程序,并且未对$\sdf$和$\df$使用相同的拟合程序。我们提出了一种\emph{修正搜索自由度}($\msdf$),以直接解释在原始空间或增广空间中的搜索成本。我们针对多种拟合程序检查了这一定义,包括经典线性回归、样条方法、自适应回归(最佳子集和lasso)、回归树以及多元自适应回归样条(MARS)。在许多$\sdf$适用的场景中,$\msdf$可简化为$\sdf$。然而,对于某些程序(如lasso),$\msdf$为搜索成本提供了新的视角。对于某些复杂程序(如MARS),$\df$在模型拟合过程中已被预先确定,但最终拟合程序的$\df$可能与预先确定的$\df$不同。为研究这一差异,我们引入了\emph{名义}$\df$和\emph{实际}$\df$的概念,并定义了\emph{自一致性}属性,即当这两个$\df$之间没有差距时发生。我们针对MARS提出了一种校正程序,以对齐这两个$\df$,并通过广泛的模拟和两个实际数据应用证明了改进的拟合性能。
英文摘要:
Model degrees of freedom ($\df$) is a fundamental concept in statistics because it quantifies the flexibility of a fitting procedure and is indispensable in model selection. To investigate the gap between $\df$ and the number of independent variables in the fitting procedure, \textcite{tibshiraniDegreesFreedomModel2015} introduced the \emph{search degrees of freedom} ($\sdf$) concept to account for the search cost during model selection. However, this definition has two limitations: it does not consider fitting procedures in augmented spaces and does not use the same fitting procedure for $\sdf$ and $\df$. We propose a \emph{modified search degrees of freedom} ($\msdf$) to directly account for the cost of searching in either original or augmented spaces. We check this definition for various fitting procedures, including classical linear regressions, spline methods, adaptive regressions (the best subset and the lasso), regression trees, and multivariate adaptive regression splines (MARS). In many scenarios when $\sdf$ is applicable, $\msdf$ reduces to $\sdf$. However, for certain procedures like the lasso, $\msdf$ offers a fresh perspective on search costs. For some complex procedures like MARS, the $\df$ has been pre-determined during model fitting, but the $\df$ of the final fitted procedure might differ from the pre-determined one. To investigate this discrepancy, we introduce the concepts of \emph{nominal} $\df$ and \emph{actual} $\df$, and define the property of \emph{self-consistency}, which occurs when there is no gap between these two $\df$'s. We propose a correction procedure for MARS to align these two $\df$'s, demonstrating improved fitting performance through extensive simulations and two real data applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。