arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.14341cs.LG
收藏

HoTPP基准测试:我们在长时域事件预测方面做得好吗?

HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?

Sber AI Lab(Sber AI实验室);Skoltech(斯克里普钦科技学院)

Ivan Karpukhin, Foma Shipilov, Andrey Savchenko

AI总结:

针对长时域事件预测缺乏基准的问题,提出HoTPP基准,引入T-mAP指标,发现现代MTPP方法常不如统计基线且存在模式崩溃。

摘要:

在给定时间范围内预测多个未来事件对于金融、零售、社交网络和医疗保健等应用至关重要。标记时间点过程(MTPP)为建模事件的时间和标签提供了一个原则性框架。然而,大多数现有研究仅关注预测下一个事件,导致长时域预测在很大程度上未被充分探索。为解决这一空白,我们引入了HoTPP,这是第一个专门设计用于严格评估长时域预测的基准测试。我们指出了广泛使用的评估指标中的缺陷,提出了一种具有理论基础的T-mAP指标,提供了强大的统计基线,并提供了流行模型的高效实现。我们的实证结果表明,现代MTPP方法通常不如简单的统计基线。此外,我们分析了预测序列的多样性,发现大多数方法表现出模式崩溃。最后,我们分析了自回归和基于强度的损失对预测质量的影响,并概述了未来研究的有前景方向。HoTPP源代码、超参数和完整评估结果可在GitHub上获取。

英文摘要:

Forecasting multiple future events within a given time horizon is essential for applications in finance, retail, social networks, and healthcare. Marked Temporal Point Processes (MTPP) provide a principled framework to model both the timing and labels of events. However, most existing research focuses on predicting only the next event, leaving long-horizon forecasting largely underexplored. To address this gap, we introduce HoTPP, the first benchmark specifically designed to rigorously evaluate long-horizon predictions. We identify shortcomings in widely used evaluation metrics, propose a theoretically grounded T-mAP metric, present strong statistical baselines, and offer efficient implementations of popular models. Our empirical results demonstrate that modern MTPP approaches often underperform simple statistical baselines. Furthermore, we analyze the diversity of predicted sequences and find that most methods exhibit mode collapse. Finally, we analyze the impact of autoregression and intensity-based losses on prediction quality, and outline promising directions for future research. The HoTPP source code, hyperparameters, and full evaluation results are available at GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑