arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.05176cs.LGcs.ROstat.ML
收藏

迈向分布不变回归的第一步

A First Step Towards Distribution Invariant Regression Metrics

International Computer Science Institute(国际计算机科学研究所);University of California, Berkeley(加州大学伯克利分校);University of Bremen(不来梅大学);DFKI GmbH(德国人工智能研究中心)

Mario Michael Krell, Bilal Wehbe

摘要:

回归评估已经进行了几十年。一些指标已被确定为对数据的移动和缩放具有鲁棒性,但考虑到数据的不同分布,更难以解决(不平衡问题),尽管它在很大程度上影响了不同数据集评价之间的可比性。在分类中,人们反复指出,像F-Measure和Accuracy这样的性能指标高度依赖于类分布,并且具有不同分布的不同数据集之间的比较是不可能的。我们发现,同样的问题存在于回归。在机器人应用中的测距参数的分布可以例如在不同的记录会话之间很大程度上变化。在这里,我们需要回归算法,要么对所有函数值执行同样好,要么专注于某些边界区域,如高速。这一点必须反映在评价指标中。我们建议修改已建立的回归指标的函数值$Y$或样本$X$使用自动调整高斯核密度估计的逆分布加权。我们在可重复实验中的合成和机器人数据上显示,经典度量的行为是错误的,而我们的新度量对变化的分布不太敏感,特别是在$X$中的边缘分布进行校正时。我们的新评估概念可以比较具有不同分布的不同数据集之间的结果。此外,它可以揭示回归算法对过度代表的目标值的过度拟合。结果,由于我们校正的度量,非过拟合回归算法将更有可能被选择。

英文摘要:

Regression evaluation has been performed for decades. Some metrics have been identified to be robust against shifting and scaling of the data but considering the different distributions of data is much more difficult to address (imbalance problem) even though it largely impacts the comparability between evaluations on different datasets. In classification, it has been stated repeatedly that performance metrics like the F-Measure and Accuracy are highly dependent on the class distribution and that comparisons between different datasets with different distributions are impossible. We show that the same problem exists in regression. The distribution of odometry parameters in robotic applications can for example largely vary between different recording sessions. Here, we need regression algorithms that either perform equally well for all function values, or that focus on certain boundary regions like high speed. This has to be reflected in the evaluation metric. We propose the modification of established regression metrics by weighting with the inverse distribution of function values $Y$ or the samples $X$ using an automatically tuned Gaussian kernel density estimator. We show on synthetic and robotic data in reproducible experiments that classical metrics behave wrongly, whereas our new metrics are less sensitive to changing distributions, especially when correcting by the marginal distribution in $X$. Our new evaluation concept enables the comparison of results between different datasets with different distributions. Furthermore, it can reveal overfitting of a regression algorithm to overrepresented target values. As an outcome, non-overfitting regression algorithms will be more likely chosen due to our corrected metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑