arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.10327stat.MLcs.LG
收藏

基于随机矩阵理论的多任务回归分析及其在时间序列预测中的应用

Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting

Paris Descartes University(巴黎笛卡尔大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳))

Romain Ilbert, Malik Tiomoko, Cosme Louart, Ambroise Odonnat, Vasilii Feofanov, Themis Palpanas, Ievgen Redko

AI总结:

本文利用随机矩阵理论构建高维多任务回归框架,推导线性模型闭式解并一致估计训练与测试误差,实验证明其可借助多变量信息提升单变量时间序列预测性能。

摘要:

本文提出了一种新颖的多任务回归理论框架,在高维、非高斯数据分布下应用随机矩阵理论以提供精确的性能估计。我们将多任务优化问题表述为一种正则化技术,使单任务模型能够利用多任务学习信息。在线性模型背景下,我们推导了多任务优化的闭式解。我们的分析通过将多任务学习性能与多种模型统计量相关联,提供了有价值的见解,这些统计量包括原始数据协方差、信号生成超平面、噪声水平以及数据集的规模和数量。最后,我们提出了训练误差和测试误差的一致估计,从而为多任务回归场景中的超参数优化提供稳健基础。在合成数据集和真实数据集上针对回归及多变量时间序列预测进行的实验验证表明,将我们的方法纳入训练损失并由此利用多变量信息,可改进单变量模型。

英文摘要:

In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑