基于Softmax近似的最优动态策略推断
Inference on Optimal Dynamic Policies via Softmax Approximation
AI总结：
针对离线最优动态策略估计中置信区间构建难的问题，提出结合半参数推断、g-估计与三角数组中心极限定理的softmax近似方法，可实现真实最优策略的有效推断，且可扩展至有限时域场景。
摘要：
从离线数据中估计最优动态策略是动态决策领域的一个基础问题。在因果推断语境下，该问题被称为估计最优动态治疗方案。尽管已有大量估计方法，但为最优策略的价值及与其相关的结构参数构建置信区间本质上更具难度，因为这涉及需要估计的未知量的非线性、不可微泛函。以往研究采用的子样本方法可能会降低估计质量。我们证明，对最优治疗方案采用简单的soft-max近似，配合适当快速增长的温度参数，能够对真实的最优策略实现有效推断。我们在两周期最优动态策略下验证了我们的结果，不过我们的方法可直接扩展到有限时域场景。本研究结合了半参数推断与g-估计技术，以及相应的三角数组中心极限定理，同时对softmax近似的渐近影响与渐近偏差进行了新颖的分析。
英文摘要：
Estimating optimal dynamic policies from offline data is a fundamental problem in dynamic decision making. In the context of causal inference, the problem is known as estimating the optimal dynamic treatment regime. Even though there exists a plethora of methods for estimation, constructing confidence intervals for the value of the optimal regime and structural parameters associated with it is inherently harder, as it involves non-linear and non-differentiable functionals of unknown quantities that need to be estimated. Prior work resorted to sub-sample approaches that can deteriorate the quality of the estimate. We show that a simple soft-max approximation to the optimal treatment regime, for an appropriately fast growing temperature parameter, can achieve valid inference on the truly optimal regime. We illustrate our result for a two-period optimal dynamic regime, though our approach should directly extend to the finite horizon case. Our work combines techniques from semi-parametric inference and $g$-estimation, together with an appropriate triangular array central limit theorem, as well as a novel analysis of the asymptotic influence and asymptotic bias of softmax approximations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
