arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.09750cs.LGmath.OC
收藏

基于双镜像下降的联合在线学习与决策

Joint Online Learning and Decision-making via Dual Mirror Descent

University of California, Berkeley(加州大学伯克利分校);Google DeepMind(谷歌DeepMind)

Alfonso Lobos, Paul Grigas, Zheng Wen

摘要:

我们考虑一个在线收入最大化问题在有限的时间范围内的成本上下限。在每个周期,代理接收i.i.d.采样的上下文向量。从未知的分布,并需要作出决定自适应。收益和成本函数取决于上下文向量以及要学习的一些固定但可能未知的参数向量。我们提出了一个新的离线基准和一个新的算法,混合了在线双镜下降计划与通用参数学习过程。当参数向量是已知的,我们证明了一个$O(\sqrt{T})$遗憾的结果,以及$O(\sqrt{T})$约束上可能的约束违反。当参数是未知的,必须学习,我们证明了遗憾和约束违反的总和以前的$O(\sqrt{T})$条款加上条款,直接依赖于学习过程的收敛。

英文摘要:

We consider an online revenue maximization problem over a finite time horizon subject to lower and upper bounds on cost. At each period, an agent receives a context vector sampled i.i.d. from an unknown distribution and needs to make a decision adaptively. The revenue and cost functions depend on the context vector as well as some fixed but possibly unknown parameter vector to be learned. We propose a novel offline benchmark and a new algorithm that mixes an online dual mirror descent scheme with a generic parameter learning process. When the parameter vector is known, we demonstrate an $O(\sqrt{T})$ regret result as well an $O(\sqrt{T})$ bound on the possible constraint violations. When the parameter is not known and must be learned, we demonstrate that the regret and constraint violations are the sums of the previous $O(\sqrt{T})$ terms plus terms that directly depend on the convergence of the learning process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑