通过锚定实现模型一致性
Model Agreement via Anchoring
AI总结：
本文提出了一种基于锚定的技术，用于证明四种常见机器学习算法在独立模型不一致度上的界限，通过调整训练参数减少模型预测的不一致性。
摘要：
许多研究旨在控制模型不一致程度——即两个机器学习模型在预测上的不一致程度。我们采用一个简单且标准的模型不一致度定义，用于实值预测问题，即两个在独立样本上训练的模型预测之间的期望平方差异，不涉及任何训练过程的协调。我们希望能够通过一些自然的训练参数（或参数）将不一致度驱动到零，使用可应用于现有训练方法的分析。我们开发了一种简单的通用技术，用于基于对两个模型平均值的锚定来证明独立模型不一致度的界限。然后我们将这种技术应用于证明四种常用机器学习算法的不一致度界限：（1）任意模型类上的堆叠聚合（其中不一致度随着堆叠的模型数量k而驱动到零）（2）梯度提升（其中不一致度随着迭代次数k而驱动到零）（3）具有架构搜索的神经网络训练（其中不一致度随着被优化的架构大小n而驱动到零）以及（4）固定深度的所有回归树上的回归树训练（其中不一致度随着树架构深度d而驱动到零）。为了清晰起见，我们首先在一维回归的平方误差损失设置中推导了初始的界限，但随后证明所有结果都可推广到多维回归和任何强凸损失函数。
英文摘要：
Numerous lines of aim to control $\textit{model disagreement}$ -- the extent to which two machine learning models disagree in their predictions. We adopt a simple and standard notion of model disagreement in real-valued prediction problems, namely the expected squared difference in predictions between two models trained on independent samples, without any coordination of the training processes. We would like to be able to drive disagreement to zero with some natural parameter(s) of the training procedure using analyses that can be applied to existing training methodologies. We develop a simple general technique for proving bounds on independent model disagreement based on $\textit{anchoring}$ to the average of two models within the analysis. We then apply this technique to prove disagreement bounds for four commonly used machine learning algorithms: (1) stacked aggregation over an arbitrary model class (where disagreement is driven to 0 with the number of models $k$ being stacked) (2) gradient boosting (where disagreement is driven to 0 with the number of iterations $k$) (3) neural network training with architecture search (where disagreement is driven to 0 with the size $n$ of the architecture being optimized over) and (4) regression tree training over all regression trees of fixed depth (where disagreement is driven to 0 with the depth $d$ of the tree architecture). For clarity, we work out our initial bounds in the setting of one-dimensional regression with squared error loss -- but then show that all of our results generalize to multi-dimensional regression with any strongly convex loss.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
