arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.13373cs.LG
收藏

基于局部度量学习的连续行为情境强盗的非策略评估

Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions

KAIST(韩国科学技术院);UC Berkeley(加州大学伯克利分校);Korea Univ.(高丽大学);Gauss Labs Inc.(高斯实验室公司);Hanyang Univ.(汉阳大学);KIAS(韩国高等研究院)

Haanvid Lee, Jongmin Lee, Yunseon Choi, Wonseok Jeon, Byung-Jun Lee, Yung-Kyun Noh, Kee-Eung Kim

摘要:

我们考虑局部核度量学习的离线政策评估(OPE)的确定性政策的上下文土匪连续的行动空间。我们的工作是出于实际情况下,目标政策需要是确定性的,由于域的要求,如处方的治疗剂量和持续时间在医学。虽然重要性采样(IS)为OPE提供了一个基本原则,但它对于具有连续动作的确定性目标策略是不适定的。我们的主要思想是放松目标策略,并将问题作为基于内核的估计,在这里我们学习内核度量,以最小化整体均方误差(MSE)。我们提出了一个分析解决方案的最佳指标,偏差和方差分析的基础上。而以前的工作已被限制到标量动作空间或内核带宽的选择,我们的工作更进一步能够向量动作空间和度量优化。我们表明,我们的估计是一致的,并显着降低了MSE相比,基线OPE方法,通过在各个领域的实验。

英文摘要:

We consider local kernel metric learning for off-policy evaluation (OPE) of deterministic policies in contextual bandits with continuous action spaces. Our work is motivated by practical scenarios where the target policy needs to be deterministic due to domain requirements, such as prescription of treatment dosage and duration in medicine. Although importance sampling (IS) provides a basic principle for OPE, it is ill-posed for the deterministic target policy with continuous actions. Our main idea is to relax the target policy and pose the problem as kernel-based estimation, where we learn the kernel metric in order to minimize the overall mean squared error (MSE). We present an analytic solution for the optimal metric, based on the analysis of bias and variance. Whereas prior work has been limited to scalar action spaces or kernel bandwidth selection, our work takes a step further being capable of vector action spaces and metric optimization. We show that our estimator is consistent, and significantly reduces the MSE compared to baseline OPE methods through experiments on various domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑