无限记忆逻辑预测的极小极大最优遗憾界

中文摘要 AI 辅助

我们确定了具有真正无限输入记忆的有限字母表、外生驱动源的极小极大累积对数损失遗憾：独立Rademacher输入$(U_t)$被顺序观测，下一个二元标记的对数几率（logit）为$\u0024\sum_{j\ge1}\theta_jU_{t+1-j}$，未知系数满足可求和包络$|\theta_j|\le r_j$，$\sum_jr_j\le B$。在时间范围$T$内，滞后$j$最多能将对数几率移动$r_j$，且仅在$n_{T,j}=(T-j+1)_+$轮中被使用，这两个限制结合为和$\Gamma_T(r)=\sum_{j\le T}\log(1+n_{T,j}r_j^{2})$。一个坐标局部化的贝叶斯混合体对每个可求和包络达到$R_T(r)\le C\Gamma_T(r)$，其中$C$是普适常数。我们的主要结果是对典型指数和多项式包络的匹配非渐近逆界；其新成分包括：针对具有外生随机设计的逻辑实验的模块化有限样本信息界，以及通过将每个非对角Gram和表示为由森林边索引的独立Rademacher变量之和来获得重叠Toeplitz滞后矩阵的条件估计，既不需要局部渐近正态性，也不需要随机Toeplitz矩阵的谱定理。因此$\Gamma_T(r)$是这里的极小极大遗憾尺度，对于$r_j=Ae^{-\alpha j}$给出$\Theta(\alpha^{-1}\log^{2}T)$，对于$r_j=Aj^{-s}$（$s>1$）给出$\Theta(T^{1/(2s)})$——后者没有窗口截断分析所付出的额外$(\log T)^{1-1/(2s)}$因子。我们还表明记忆衰减不能决定遗憾，并且一个轮廓缩放的在线牛顿预测器在每轮多项式时间内达到$O_B(\Gamma_T(r))$。

英文摘要

We determine the minimax cumulative log-loss regret of a finite-alphabet, exogenously driven source with genuinely infinite input memory: independent Rademacher inputs $(U_t)$ are observed sequentially and the next binary mark has logit $\sum_{j\ge1}θ_jU_{t+1-j}$, the unknown coefficients obeying a summable envelope $|θ_j|\le r_j$, $\sum_jr_j\le B$. At horizon $T$, lag $j$ can move the logit by at most $r_j$ and is exercised in only $n_{T,j}=(T-j+1)_+$ rounds, and the two limitations combine into the sum $Γ_T(r)=\sum_{j\le T}\log(1+n_{T,j}r_j^{2})$. One coordinate-localised Bayesian mixture achieves $R_T(r)\le CΓ_T(r)$ for \emph{every} summable envelope with $C$ universal. Our main result is a matching nonasymptotic converse for the canonical exponential and polynomial envelopes; its new ingredients are a modular finite-sample information bound for logistic experiments with an exogenous random design, and a conditioning estimate for the overlapping Toeplitz lag matrix obtained by exhibiting each off-diagonal Gram sum as a sum of independent Rademacher variables indexed by the edges of a forest, needing neither local asymptotic normality nor any spectral theorem for random Toeplitz matrices. So $Γ_T(r)$ is the minimax regret scale here, giving $Θ(α^{-1}\log^{2}T)$ for $r_j=Ae^{-αj}$ and $Θ(T^{1/(2s)})$ for $r_j=Aj^{-s}$, $s>1$ --- the latter without the extra $(\log T)^{1-1/(2s)}$ factor any window-truncation analysis pays. We also show memory decay cannot determine regret, and that a profile-scaled online Newton predictor attains $O_B(Γ_T(r))$ in polynomial time per round.

发表机构

• Purdue University（普渡大学）

机构由 AI 辅助整理，请以论文原文为准。