arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.07148cs.LGcs.AIcs.ITmath.IT
收藏

基于日志数据的半监督批量学习

Semi-supervised Batch Learning From Logged Data

The Alan Turing Institute(艾伦·图灵研究所);Sharif University of Technology(谢里夫理工大学);University College London(伦敦大学学院)

Gholamali Aminian, Armin Behnamnia, Roberto Vega, Laura Toni, Chengchun Shi, Hamid R. Rabiee, Omar Rivasplata, Miguel R. D. Rodrigues

AI总结:

针对日志数据中部分样本反馈缺失的问题,提出基于反事实风险最小化框架的半监督批量学习方法,通过推导真实风险上界设计独立于反馈的正则化项,利用缺失反馈数据提升策略性能。

摘要:

Off-policy learning 方法旨在从日志数据中学习策略,日志数据包含每个样本点的上下文、动作和反馈(成本或奖励)。在本研究中,我们基于反事实风险最小化框架,该框架还假设可以访问倾向性得分。针对部分样本反馈缺失的问题,我们提出了相应的学习方法,使得日志数据中同时包含有反馈样本和缺失反馈样本。我们将此类学习称为基于日志数据的半监督批量学习,它广泛应用于众多应用领域。为解决此类学习问题,我们推导出在逆倾向性得分估计器下真实风险的新颖上界。利用该上界,我们提出了一种基于日志数据的正则化半监督批量学习方法,其正则化项独立于反馈,因此可使用日志中的缺失反馈数据进行评估。因此,即使反馈仅存在于部分样本中,也能通过利用缺失反馈样本来学习策略。基于基准数据集的实验结果表明,与日志策略相比,这些算法能获得性能更优的策略。

英文摘要:

Off-policy learning methods are intended to learn a policy from logged data, which includes context, action, and feedback (cost or reward) for each sample point. In this work, we build on the counterfactual risk minimization framework, which also assumes access to propensity scores. We propose learning methods for problems where feedback is missing for some samples, so there are samples with feedback and samples missing-feedback in the logged data. We refer to this type of learning as semi-supervised batch learning from logged data, which arises in a wide range of application domains. We derive a novel upper bound for the true risk under the inverse propensity score estimator to address this kind of learning problem. Using this bound, we propose a regularized semi-supervised batch learning method with logged data where the regularization term is feedback-independent and, as a result, can be evaluated using the logged missing-feedback data. Consequently, even though feedback is only present for some samples, a learning policy can be learned by leveraging the missing-feedback samples. The results of experiments derived from benchmark datasets indicate that these algorithms achieve policies with better performance in comparison with logging policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑