为何线性模型在复杂且变化的世界中仍能发挥作用
Why Linear Models Can Still Work in a Complex and Changing World
AI总结:
该研究探讨了复杂变化环境中线性模型仍有效的原因,指出其源于世界复杂度超有限数据可学习范围,提出模型复杂度需结合跨环境稳定性与训练数据准确性选择。
摘要:
线性模型即便在底层机制为非线性的复杂领域也能实现良好预测。我们认为这一现象的出现是因为世界的复杂性超出了从有限、部分观测数据中可靠学习到的复杂度。灵活模型可利用曲率、阈值和交互作用拟合某一医院、时期、人群、政策环境或测量过程,但在其他环境中会失效。当这种不稳定性主要存在于非线性成分时,忽略该成分会使线性模型在新环境中具有更低的预测误差。我们将这一主张与关于简单线性规则、偏差与方差、正则化、最优线性预测、测量误差、分布偏移、不变性、Rashomon集及欠指定的研究相联系。命题1给出了一个简单的平方误差案例,其中在训练环境中学习到的非线性规则因迁移性过弱而导致目标误差上升;有限样本估计误差可强化这一结果。仅部分观测是不够的:观测变量与未观测因素之间的关联必须在不同环境间存在差异。稳定的非线性结构有利于灵活模型,而线性模型仍易受线性虚假关系的影响。因此,模型复杂度应通过跨环境稳定性及训练数据准确性来选择。
英文摘要:
Linear models can predict well in complex domains even when underlying mechanisms are nonlinear. We argue that this can occur because the complexity of the world exceeds the complexity that can be learned reliably from finite, partially observed data. Flexible models can exploit curvature, thresholds, and interactions that fit one hospital, period, population, policy setting, or measurement process but fail elsewhere. When this instability lies mainly in the nonlinear component, omitting it can give linear models lower prediction error in a new environment. We connect this claim to work on simple linear rules, bias and variance, regularization, best linear prediction, measurement error, distribution shift, invariance, Rashomon sets, and underspecification. Proposition 1 gives a simple squared-error case in which a nonlinear rule learned in the training environment raises target error because it transfers too weakly; finite-sample estimation error can strengthen this result. Partial observation alone is not enough: the link between observed variables and unobserved factors must vary across environments. Stable nonlinear structure favors flexible models, and linear models remain vulnerable to linear spurious relationships. Model complexity should therefore be chosen by cross-environment stability as well as training-data accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。