可重放马尔可夫链
Markov Chains with Rewinding
AI总结:
本文研究了可重放马尔可夫链,探讨了通过重放策略区分初始状态的方法,证明非适应性策略在效率上与适应性策略有多项式差距。
摘要:
受近期子线性时间算法下界研究进展的启发,本文引入并研究了一类新的随机算法过程,称为可重放马尔可夫链。在该设定中,算法通过策略性地重放马尔可夫链到先前状态与部分可观测的马尔可夫随机过程互动。根据应用需求,这可能使过程更快达到期望状态,或使代理更高效地学习或测试马尔可夫链的属性。我们研究了在给定部分可观测马尔可夫链中识别初始状态的任务。分析该问题在特定马尔可夫链中的情况是上述引用工作的核心成分,我们旨在在本文中系统化这一分析。我们的第一个结果是,任何可通过任何重放策略区分的两个状态也可通过非适应性重放策略(其重放选择在观察任何链结果之前确定)区分。因此,尽管重放策略可以证明比被动策略(不回退到先前状态的策略)更强大,但在没有效率考虑的情况下,适应性并未为重放策略提供额外的权力。然而,当引入自然的效率度量,即查询复杂性(即识别可区分状态所需的观察次数)时,差异变得明显。我们的第二个主要贡献是量化这一效率差距。我们提出了一种非适应性重放策略,其查询复杂性与最优(适应性)策略的复杂性在多项式范围内,并证明这种多项式损失在一般情况下是必要的。
英文摘要:
Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic processes that we call Markov Chains with Rewinding. In this setting, an algorithm interacts with a (partially observable) Markovian random evolution by strategically rewinding the Markov chain to previous states. Depending on the application, this may lead the evolution to desired states faster, or allow the agent to efficiently learn or test properties of the underlying Markov chain that may be infeasible or inefficient with passive observation. We study the task of identifying the initial state in a given partially observable Markov chain. Analysis of this question in specific Markov chains is the central ingredient in the above cited works and we aim to systematize the analysis in our work. Our first result is that any pair of states distinguishable with any rewinding strategy can also be distinguished with a non-adaptive rewinding strategy (one whose rewinding choices are determined before observing any outcomes of the chain). Therefore, while rewinding strategies can be shown to be strictly more powerful than passive strategies (those that do not rewind back to previous states), adaptivity does not give additional power to a rewinding strategy in the absence of efficiency considerations. The difference becomes apparent however when we introduce a natural efficiency measure, namely the query complexity (i.e., the number of observations they need to identify distinguishable states). Our second main contribution is to quantify this efficiency gap. We present a non-adaptive rewinding strategy whose query complexity is within a polynomial of that of the optimal (adaptive) strategy, and show that such a polynomial loss is necessary in general.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。