arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.04602math.STcs.ITmath.ITstat.MEstat.TH
收藏

在线分位数回归

Online Quantile Regression

Yinan Shen, Dong Xia, Wen-Xin Zhou

AI总结:

本文针对在线分位数回归中特征增长、范围未知及内存受限的问题,采用随机次梯度下降法,实现了长期最优估计,并获得了仅对初始误差短期敏感的指数型浓度不等式与最优遗憾率。

摘要:

本文解决了在分位数回归框架内整合顺序到达数据所面临的挑战,其中特征数量允许随观测数量增长,时间范围未知,且内存有限。我们采用随机次梯度下降法来最小化经验检查损失,并研究其统计性质和遗憾性能。在我们的分析中,我们揭示了基于单个观测与基于批次观测更新迭代之间的微妙相互作用,展示了每种情形下不同的正则性性质。我们的方法确保了无论选择何种更新策略,都能实现长期最优估计。重要的是,我们的贡献超越了先前的工作,实现了指数型浓度不等式,并获得了仅对初始误差表现出短期敏感性的最优遗憾和误差率。我们研究的一个关键见解是精细的统计分析,以及揭示了适当的步长方案能显著减轻初始误差对后续误差和遗憾的影响。这强调了随机次梯度下降在处理初始不确定性时的稳健性,突出了其在顺序数据到达引入关于时间范围和总观测数量不确定性的场景中的有效性。此外,当初始误差率得到良好控制时,短期误差率与长期最优性之间存在权衡。由于缺乏对平方损失的精细统计分析,我们也简要讨论了其性质和适当的方案。广泛的模拟支持了我们的理论发现。

英文摘要:

This paper addresses the challenge of integrating sequentially arriving data within the quantile regression framework, where the number of features is allowed to grow with the number of observations, the horizon is unknown, and memory is limited. We employ stochastic sub-gradient descent to minimize the empirical check loss and study its statistical properties and regret performance. In our analysis, we unveil the delicate interplay between updating iterates based on individual observations versus batches of observations, revealing distinct regularity properties in each scenario. Our method ensures long-term optimal estimation irrespective of the chosen update strategy. Importantly, our contributions go beyond prior works by achieving exponential-type concentration inequalities and attaining optimal regret and error rates that exhibit only \textsf{ short-term} sensitivity to initial errors. A key insight from our study is the delicate statistical analyses and the revelation that appropriate stepsize schemes significantly mitigate the impact of initial errors on subsequent errors and regrets. This underscores the robustness of stochastic sub-gradient descent in handling initial uncertainties, emphasizing its efficacy in scenarios where the sequential arrival of data introduces uncertainties regarding both the horizon and the total number of observations. Additionally, when the initial error rate is well-controlled, there is a trade-off between short-term error rate and long-term optimality. Due to the lack of delicate statistical analysis for squared loss, we also briefly discuss its properties and proper schemes. Extensive simulations support our theoretical findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑