arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.00265cs.LGcs.CV
收藏

机器学习模型的离群值鲁棒训练

Outlier-Robust Training of Machine Learning Models

Massachusetts Institute of Technology(麻省理工学院);Laboratory of Information & Decision Systems(信息与决策系统实验室)

Rajat Talak, Charis Georgiou, Jingnan Shi, Luca Carlone

AI总结:

本文通过修改Black-Rangarajan对偶性统一了M估计与风险最小化两类鲁棒损失设计,提出自适应交替算法(AAA),利用鲁棒损失核σ扩大收敛区域,并在回归、分类和神经场景重建中验证了有效性。

摘要:

在存在离群值的情况下对机器学习模型进行鲁棒训练已在多个领域引起关注。使用鲁棒损失是一种流行的方法,并且已知能够减轻离群值的影响。我们揭示了两类在设计鲁棒损失方式上已经分化的文献:一类使用M估计,在机器人学和计算机视觉中很流行;另一类使用风险最小化框架,在深度学习中很流行。我们首先证明,对Black-Rangarajan对偶性的简单修改提供了一个统一的视角。修改后的对偶性引出了鲁棒损失核σ的定义,该定义同时被两类文献中的鲁棒损失所满足。其次,利用修改后的对偶性,我们提出了一种用于训练带有离群值的机器学习模型的自适应交替算法(AAA)。该算法通过使用非鲁棒损失的加权版本迭代训练模型,同时在每次迭代中更新权重。通过将权重解释为内点概率,该算法增加了一种新的参数更新规则,从而避免了复杂的参数调优。第三,我们研究了自适应交替算法收敛到无离群值最优解的性质。考虑任意离群值(即不对离群值作分布假设),我们证明使用鲁棒损失核σ能够扩大收敛区域。我们通过实验展示了该算法在回归、分类和神经场景重建问题上的有效性。我们发布了实现代码:https://github.com/MIT-SPARK/ORT。

英文摘要:

Robust training of machine learning models in the presence of outliers has garnered attention across various domains. The use of robust losses is a popular approach and is known to mitigate the impact of outliers. We bring to light two literatures that have diverged in their ways of designing robust losses: one using M-estimation, which is popular in robotics and computer vision, and another using a risk-minimization framework, which is popular in deep learning. We first show that a simple modification of the Black-Rangarajan duality provides a unifying view. The modified duality brings out a definition of a robust loss kernel $σ$ that is satisfied by robust losses in both the literatures. Secondly, using the modified duality, we propose an Adaptive Alternation Algorithm (AAA) for training machine learning models with outliers. The algorithm iteratively trains the model by using a weighted version of the non-robust loss, while updating the weights at each iteration. The algorithm is augmented with a novel parameter update rule by interpreting the weights as inlier probabilities, and obviates the need for complex parameter tuning. Thirdly, we investigate convergence of the adaptive alternation algorithm to outlier-free optima. Considering arbitrary outliers (i.e., with no distributional assumption on the outliers), we show that the use of robust loss kernels σ increases the region of convergence. We experimentally show the efficacy of our algorithm on regression, classification, and neural scene reconstruction problems. We release our implementation code: https://github.com/MIT-SPARK/ORT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑