时间的问题：重新思考使用时间分割对谣言检测基准的评估
It's about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits
摘要：
随着时间的推移，新的事件出现，影响社交媒体上的谣言话题。目前的谣言检测基准使用随机分割作为训练、开发和测试集，这通常会导致主题重叠。因此，由于时间概念漂移，在随机分割上训练的模型可能在以前看不见的主题上的谣言分类上表现不佳。在本文中，我们提供了一个重新评估的分类模型的四个流行的谣言检测基准考虑时间顺序，而不是随机分裂。我们的实验结果表明，使用随机分割可以显著高估所有数据集和模型的预测性能。因此，我们建议谣言检测模型应该始终使用时间分割进行评估，以最大限度地减少主题重叠。
英文摘要：
New events emerge over time influencing the topics of rumors in social media. Current rumor detection benchmarks use random splits as training, development and test sets which typically results in topical overlaps. Consequently, models trained on random splits may not perform well on rumor classification on previously unseen topics due to the temporal concept drift. In this paper, we provide a re-evaluation of classification models on four popular rumor detection benchmarks considering chronological instead of random splits. Our experimental results show that the use of random splits can significantly overestimate predictive performance across all datasets and models. Therefore, we suggest that rumor detection models should always be evaluated using chronological splits for minimizing topical overlaps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
