基于单纯形扰动-动作策略的存储系统在线约束控制

中文摘要 AI 辅助

我们研究了一个标量存储系统的在线控制问题，该系统面临非负对抗性资源到达和状态相关的动作约束。对抗性的、时变的成本同时取决于状态和动作。在每个时刻，控制器在当前资源到达和成本揭示之前选择一个可行的动作。为了处理可行性与在线学习之间的耦合，我们引入了单纯形扰动-动作控制（SDAC），其策略具有$H$个非负参数，这些参数之和至多为1。每个固定的SDAC策略通过构造保证可行性。我们的在线SDAC控制器使用熵在线镜像下降更新这些参数，同时沿时变轨迹保持可行性。我们证明了相对于最佳固定SDAC策略的遗憾界为$O\\!\left(\sqrt{T\log(H+1)}\right)$，并给出了一个极小极大下界，表明对$T$的依赖是紧的。我们还引入了无限记忆SDAC策略，该类策略包含所有可行的固定比例策略。SDAC策略以随$H$几何递减的误差逼近该类策略。对于合适的$H=\Theta(\log T)$选择，相对于无限记忆SDAC策略的遗憾为$O\\!\left(\sqrt{T\log\log T}\right)$。该框架适用于能量收集电池和其他存储受限的资源系统。

英文摘要

We study online control of a scalar storage system with nonnegative adversarial resource arrivals and state-dependent action constraints. The adversarial, time-varying cost depends on both state and action. At each time, the controller selects a feasible action before the current resource arrival and cost are revealed. To handle the coupling between feasibility and online learning, we introduce Simplex Disturbance-Action Control (SDAC), whose policies have $H$ nonnegative parameters summing to at most one. Every fixed SDAC policy is feasible by construction. Our online SDAC controller updates these parameters using entropic online mirror descent while preserving feasibility along the time-varying trajectory. We prove a regret bound of $O\!\left(\sqrt{T\log(H+1)}\right)$ relative to the best fixed SDAC policy and a minimax lower bound showing that the dependence on $T$ is tight. We also introduce infinite-memory SDAC policies, which include every feasible fixed-fraction policy. SDAC policies approximate this class with an error that decreases geometrically with $H$. For a suitable choice $H=Θ(\log T)$, the resulting regret against infinite-memory SDAC policies is $O\!\left(\sqrt{T\log\log T}\right)$. The framework applies to energy-harvesting batteries and other storage-constrained resource systems.

发表机构

• University of Southern California（南加州大学）

机构由 AI 辅助整理，请以论文原文为准。