发表机构
University of Copenhagen（哥本哈根大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对遗忘可实现对手，证明私有在线学习与预测的样本复杂度被时间范围因子分离，给出学习下界和预测上界，并填补先前下界空白。
AI 中文摘要
我们研究了在遗忘可实现对手下，差分私有在线学习和在线预测的失误界。在线学习要求学习器在每个时间步发布一个假设，而在线预测中，学习器仅需做出预测而无需发布假设。利用一个针对私有在线学习的新下界和针对私有预测的上界，我们证明了对于每个有限 Littlestone 维数 $d$ 的类别，这两个问题的样本复杂度被一个随时间范围增长的因子所分离。首先，我们证明每个 $\br{\epsilon,\delta}$-私有在线学习器在长度为 $T$ 的确定性可实现流上，其失误界至少为 $\bE\bs{M_T}=\Om{\frac d\epsilon \log\br{ T}^{2/3}}$。特别地，这是先前工作[SR22,DSS24,LWY24]中留下的范围 $1/T<\delta<1/\log T$ 内的首个非平凡下界。其次，我们证明对于每个 Littlestone 维数为 $d$ 的类别，存在一个 $(\u03b5,\u03b4)$-联合私有预测器，其预期失误数至多为 $2^{2^{cd^2}}\epsilon^{-2}\log^2\br{2/\br{\epsilon\delta}}$，且与 $T$ 无关，其中 $c>0$ 为某个绝对常数。因此，对于每个固定的有限 Littlestone 维数类别，当 $\delta=\Theta\br{1/\log T}$ 时，私有学习需要 $\Om{\br{\log T}^{2/3}}$ 次预期失误，而私有预测允许 $\bigO{\br{\log\log T}^2}$ 次失误。
英文摘要
We study mistake bounds for differentially private online learning and online prediction under oblivious realisable adversaries. Online learning requires the learner to release a hypothesis at each time step whereas in online prediction, the learner only needs to make predictions without releasing a hypothesis. Using a novel lower bound for private online learning and an upper bound for private prediction, we show that the sample complexity of these two problems are separated by a factor that grows with the time horizon for every class of finite Littlestone dimension $d$. First, we prove that every $(ε,δ)$-private online learner has a deterministic realisable stream of length $T$ on which the mistake bound is at least $\mathbb E [M_T]= O\left(\frac dε\log^{2/3} T\right)$. In particular, this is the first non-trivial lower in the range $1/T<δ<1/\log T)$ left open in earlier works[SR22,DSS24,LWY24]. Second, we prove that for every class of of Littlestone dimension $d$, there exists an $(ε,δ)$-jointly private predictor with at most $2^{2^{cd^2}}ε^{-2}\log^2(2/(εδ))$ expected mistakes, independently of $T$, for some absolute constant $c>0$. Thus, for every fixed class of finite Littlestone dimension when $δ=Θ(1/\log T)$, private learning requires $O\left(\log^{2/3} T\right)$ expected mistakes, whereas private prediction admits $O(\log\log T^2)$.
