arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.04274cs.LGcs.AI
收藏

在数百篇评估图形表示的论文中使用不切实际的预测

On the Use of Unrealistic Predictions in Hundreds of Papers Evaluating Graph Representations

National Taiwan University(国立台湾大学);National Chengchi University(国立政治大学);University of Southern California(南加州大学);ASUS Intelligent Cloud Services(华硕智能云服务)

Li-Chung Lin, Cheng-Hung Liu, Chih-Ming Chen, Kai-Chin Hsu, I-Feng Wu, Ming-Feng Tsai, Chih-Jen Lin

摘要:

在机器学习中,使用基本事实进行预测听起来像是一种矛盾修饰法。然而,这样一个不切实际的设置被用在数百,如果不是成千上万的论文在该地区找到图形表示。为了评估多标签问题的节点分类使用所获得的表示,许多作品假设在预测阶段,每个测试实例的标签的数量是已知的。在实践中,这样的地面真相信息很少,但我们指出,这样一个不适当的设置,现在是无处不在的,在这个研究领域。我们详细调查了为什么会出现这种情况。我们的分析表明,由于信息不切实际,业绩可能被高估。为了了解为什么没有使用合适的预测,我们确定了应用一些多标签技术的困难。为了在未来的研究中使用,我们提出了简单而有效的设置,而不使用实际上未知的信息。最后,我们借此机会进行公平和认真的比较主要的图表示学习方法的多标签节点分类。

英文摘要:

Prediction using the ground truth sounds like an oxymoron in machine learning. However, such an unrealistic setting was used in hundreds, if not thousands of papers in the area of finding graph representations. To evaluate the multi-label problem of node classification by using the obtained representations, many works assume in the prediction stage that the number of labels of each test instance is known. In practice such ground truth information is rarely available, but we point out that such an inappropriate setting is now ubiquitous in this research area. We detailedly investigate why the situation occurs. Our analysis indicates that with unrealistic information, the performance is likely over-estimated. To see why suitable predictions were not used, we identify difficulties in applying some multi-label techniques. For the use in future studies, we propose simple and effective settings without using practically unknown information. Finally, we take this chance to conduct a fair and serious comparison of major graph-representation learning methods on multi-label node classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑