不加反思地使用表格数据仓库可能损害研究质量
Unreflected Use of Tabular Data Repositories Can Undermine Research Quality
AI总结:
本文指出不加反思地使用表格数据仓库可能损害研究质量,并以 OpenML 为例揭示次优模型选择、忽视强基线和不当预处理三类陷阱,进而探讨数据仓库如何防止不当使用并提升实证研究质量。
摘要:
数据仓库已积累了大量来自不同领域的表格数据集。机器学习研究者正积极使用这些数据集来评估新方法。因此,数据仓库在表格数据研究中具有重要地位。它们不仅托管数据集,还提供如何在监督学习任务中使用这些数据集的信息。在本文中,我们认为,尽管在可用性方面取得了巨大成就,但不加反思地使用数据仓库中的数据集可能已导致研究质量和科学严谨性下降。我们给出了近期一些重要研究中的例子,用以说明对 OpenML(一个大型表格数据仓库)中数据集的不当使用。我们的示例有助于数据仓库的使用者避免落入以下陷阱:(1)使用次优的模型选择策略,(2)忽视强基线,以及(3)进行不适当的预处理。针对这些问题,我们讨论了数据仓库如何防止数据集被不当使用,并成为提升实证研究整体质量的基石的可行方案。
英文摘要:
Data repositories have accumulated a large number of tabular datasets from various domains. Machine Learning researchers are actively using these datasets to evaluate novel approaches. Consequently, data repositories have an important standing in tabular data research. They not only host datasets but also provide information on how to use them in supervised learning tasks. In this paper, we argue that, despite great achievements in usability, the unreflected usage of datasets from data repositories may have led to reduced research quality and scientific rigor. We present examples from prominent recent studies that illustrate the problematic use of datasets from OpenML, a large data repository for tabular data. Our illustrations help users of data repositories avoid falling into the traps of (1) using suboptimal model selection strategies, (2) overlooking strong baselines, and (3) inappropriate preprocessing. In response, we discuss possible solutions for how data repositories can prevent the inappropriate use of datasets and become the cornerstones for improved overall quality of empirical research studies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。