在不安定多臂老虎机中最小化成本而非最大化奖励
Minimizing Cost Rather Than Maximizing Reward in Restless Multi-Armed Bandits
AI总结：
本文针对不安定多臂老虎机提出以最小化成本实现奖励阈值的约束最小化问题，证明其双准则近似版本为 PSPACE-hard，建立 Whittle 指数理论并揭示其与最大化问题的联系与启发式方法的失效，最后在真实与合成数据上比较多种启发式算法。
摘要：
不安定多臂老虎机（RMABs）为解决资源受限的最大化问题提供了一个强大的框架。然而，当限制性约束是奖励阈值而非预算时，该表述可能并不适用。我们为 RMABs 引入了一个带约束的最小化问题，该问题在实现奖励阈值的目标与最小化总成本之间取得平衡。我们证明，即使该问题的双准则近似版本也是 PSPACE-hard 的。受这一困难性结果的启发，我们为最小化问题定义了解耦问题、可索引性以及 Whittle 指数，与最大化问题中的相应概念相对应。此外，我们证明了最小化问题的 Whittle 指数可以很容易地从最大化问题的 Whittle 指数计算得到。因此，最大化问题中 RMAB 实例上的 Whittle 指数结果可以直接给出最小化问题的 Whittle 指数结果。尽管最小化问题与最大化问题之间存在相似性，但解决最小化问题并不像直接套用最大化问题启发式方法的对应版本那样简单。我们给出了一个 RMAB 实例，对于该实例，贪心 Whittle 指数启发式方法在最大化问题上能达到最优解，而相应的启发式方法在最小化问题上却产生最差的可能解。鉴于此，我们在真实数据和合成数据上提出并比较了多种用于解决最小化问题的启发式方法。我们的工作表明，继续深入研究最小化问题具有重要意义。
英文摘要：
Restless Multi-Armed Bandits (RMABs) offer a powerful framework for solving resource constrained maximization problems. However, the formulation can be inappropriate for settings where the limiting constraint is a reward threshold rather than a budget. We introduce a constrained minimization problem for RMABs that balances the goal of achieving a reward threshold while minimizing total cost. We show that even a bi-criteria approximate version of the problem is PSPACE-hard. Motivated by the hardness result, we define a decoupled problem, indexability and a Whittle index for the minimization problem, mirroring the corresponding concepts for the maximization problem. Further, we show that the Whittle index for the minimization problem can easily be computed from the Whittle index for the maximization problem. Consequently, Whittle index results on RMAB instances for the maximization problem give Whittle index results for the minimization problem. Despite the similarities between the minimization and maximization problems, solving the minimization problem is not as simple as taking direct analogs of the heuristics for the maximization problem. We give an example of an RMAB for which the greedy Whittle index heuristic achieves the optimal solution for the maximization problem, while the analogous heuristic yields the worst possible solution for the minimization problem. In light of this, we present and compare several heuristics for solving the minimization problem on real and synthetic data. Our work suggests the importance of continued investigation into the minimization problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
