基于感知器的CDF建模用于时间序列预测
Perceiver-based CDF Modeling for Time Series Forecasting
AI总结：
提出perceiver-CDF架构，结合感知器与copula注意力机制，高效建模时间序列CDF，在单/多模态基准上以不到一半计算资源实现20%性能提升。
摘要：
Transformer在时间序列数据预测中已展现出显著的功效。然而，它们对自注意力机制的广泛依赖需要大量的计算资源，从而限制了它们在不同任务中的实际应用，尤其是在多模态问题中。在这项工作中，我们提出了一种名为perceiver-CDF的新架构，用于对时间序列数据的累积分布函数（CDF）进行建模。我们的方法将感知器架构与一种针对多模态时间序列预测的基于copula的注意力机制相结合。通过利用感知器，我们的模型高效地将高维和多模态数据转换为紧凑的潜在空间，从而显著降低计算需求。随后，我们实现了一种基于copula的注意力机制来构建缺失数据的联合分布以进行预测。此外，我们提出了一种输出方差测试机制，以有效缓解预测过程中的误差传播。为了提高效率并降低复杂度，我们为局部注意力机制引入了中点推理。这使得模型能够高效地捕获附近插补样本内的依赖关系，而无需考虑所有先前的样本。在单模态和多模态基准上的实验一致表明，在利用不到一半计算资源的情况下，相较于最先进的方法实现了20%的提升。
英文摘要：
Transformers have demonstrated remarkable efficacy in forecasting time series data. However, their extensive dependence on self-attention mechanisms demands significant computational resources, thereby limiting their practical applicability across diverse tasks, especially in multimodal problems. In this work, we propose a new architecture, called perceiver-CDF, for modeling cumulative distribution functions (CDF) of time series data. Our approach combines the perceiver architecture with a copula-based attention mechanism tailored for multimodal time series prediction. By leveraging the perceiver, our model efficiently transforms high-dimensional and multimodal data into a compact latent space, thereby significantly reducing computational demands. Subsequently, we implement a copula-based attention mechanism to construct the joint distribution of missing data for prediction. Further, we propose an output variance testing mechanism to effectively mitigate error propagation during prediction. To enhance efficiency and reduce complexity, we introduce midpoint inference for the local attention mechanism. This enables the model to efficiently capture dependencies within nearby imputed samples without considering all previous samples. The experiments on the unimodal and multimodal benchmarks consistently demonstrate a 20% improvement over state-of-the-art methods while utilizing less than half of the computational resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
