连续时间强化学习中的累积治疗效应检验
Testing Cumulative Treatment Effects in Continuous Time with ODE Dynamics and Noisy Observations
AI总结：
本文提出了一种基于连续时间强化学习的方法，用于检验治疗效应的累积影响，通过平均治疗效应和价值函数估计，提高了多分辨率观测下的准确性。
摘要：
理解治疗效应随时间变化的影响是许多科学和医学研究中的基本方面。在本文中，我们介绍了一种新的方法，该方法在连续时间强化学习框架下用于检验治疗效应。具体而言，我们的方法提供了一种有效的测试，以利用平均治疗效应（ATE）来检验治疗的累积效应。平均治疗效应被定义为无限时间 horizon 上价值函数的差值，这考虑了即时和累积的治疗效应。所提出的方法在多分辨率观测设置中优于现有的检验程序，如离散时间强化学习策略，其中观测时间可以是不规则的。所提出方法的另一个优势是，它可以通过对价值函数进行连续时间估计，捕捉较短持续时间的治疗效应，并比离散时间近似方法提供更高的准确性。我们建立了所提出检验统计量的渐近正态性，并将其应用于俄亥俄州 T1DM 糖尿病数据，以评估胰岛素推注对患者血糖水平的累积治疗效应。
英文摘要：
We develop a semiparametric continuous time inferential framework for testing cumulative treatment effects, including both immediate and carryover effects. The latent state process is modeled by an ordinary differential equation (ODE) whose drift depends on state and treatment, while the observed data are noisy measurements of this latent state at irregular, multi resolution time points. We define the average treatment effect (ATE) as the difference of expected cumulative outcomes under the always treat and always control policies and establish its identifiability from observational data. More generally, the framework accommodates contrasts between arbitrary deterministic stationary treatment policies, including clinically motivated threshold policies, with the always treat versus never treat comparison as the leading special case. A two step estimation procedure first recovers the latent state trajectories and their derivatives via spline smoothing, then solves a continuous time estimating equation derived from the infinitesimal generator of the ODE. We prove asymptotic normality of the resulting test statistic under explicit rate conditions on the spline estimator. The resulting test admits asymptotically valid inference under noisy, irregularly sampled observations, providing a studentized statistic with a standard $N(0,1)$ limit. Simulations show that the proposed method demonstrates improved stability and power over discrete time alternatives in noisy and multi resolution settings, with comparable performance under clean regular grids. We apply the method to the OhioT1DM diabetes dataset to evaluate cumulative effects of insulin treatments on glucose levels.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
