arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.12491cs.LGstat.ML
收藏

具有受控泛化误差的回归训练数据子集选择

Training Data Subset Selection for Regression with Controlled Generalization Error

IIT Bombay(印度理工学院孟买分校);UT Dallas(德克萨斯大学达拉斯分校)

Durga Sivasubramanian, Rishabh Iyer, Ganesh Ramakrishnan, Abir De

摘要:

从大量的训练实例中选择数据子集是一种有效且经济的机器学习方法。然而,在较小子集上训练的模型可能表现出较差的泛化能力。在本文中,我们的目标是设计一个算法,用于选择一个子集的训练数据,使模型可以快速训练,而不会显着牺牲的准确性。更具体地说,我们专注于L2正则化回归问题的数据子集选择,并提供了一个新的问题制定,旨在尽量减少训练损失的可训练参数和训练数据的子集,受到验证集上的误差界。我们通过几项技术创新来解决这个问题。首先,我们使用原始训练问题的对偶用简化的约束来表示这个问题,并表明这种新表示的目标是单调和α-次模函数,用于各种各样的建模选择。这些属性使我们开发SELCON,一个有效的优化最小化算法的数据子集选择,承认一个近似保证,即使当训练提供了一个不完美的估计训练模型。最后,我们在几个数据集上的实验表明,SELCON比当前最先进的方法更有效地权衡了准确性和效率。

英文摘要:

Data subset selection from a large number of training instances has been a successful approach toward efficient and cost-effective machine learning. However, models trained on a smaller subset may show poor generalization ability. In this paper, our goal is to design an algorithm for selecting a subset of the training data, so that the model can be trained quickly, without significantly sacrificing on accuracy. More specifically, we focus on data subset selection for L2 regularized regression problems and provide a novel problem formulation which seeks to minimize the training loss with respect to both the trainable parameters and the subset of training data, subject to error bounds on the validation set. We tackle this problem using several technical innovations. First, we represent this problem with simplified constraints using the dual of the original training problem and show that the objective of this new representation is a monotone and alpha-submodular function, for a wide variety of modeling choices. Such properties lead us to develop SELCON, an efficient majorization-minimization algorithm for data subset selection, that admits an approximation guarantee even when the training provides an imperfect estimate of the trained model. Finally, our experiments on several datasets show that SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑