arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.11065cs.CRcs.SYeess.SY
收藏

基于层次同态加密的加密值迭代和时间差分学习

Encrypted Value Iteration and Temporal Difference Learning over Leveled Homomorphic Encryption

Jihoon Suh, Takashi Tanaka

摘要:

我们考虑了一种基于同态加密(HE)的机密云控制合成架构。我们的研究是由最近激增的数据驱动控制,如深度强化学习,其繁重的计算需求往往需要外包给第三方服务器。为了实现比部分同态加密(PHE)更大的灵活性和比完全同态加密(FHE)更少的计算开销,我们考虑了基于分层同态加密(LHE)的强化学习(RL)架构。我们首先表明,在基于模型的表格值迭代(VI)的收敛性的千金金宋(CKKS)加密方案的加密噪声的影响,可以解析有界。我们还考虑了TD(0)、SARSA(0)和Z-学习算法在CKKS方案上的安全实现,其中我们用数值方法证明了加密噪声对这些算法的影响也是最小的。

英文摘要:

We consider an architecture of confidential cloud-based control synthesis based on Homomorphic Encryption (HE). Our study is motivated by the recent surge of data-driven control such as deep reinforcement learning, whose heavy computational requirements often necessitate an outsourcing to the third party server. To achieve more flexibility than Partially Homomorphic Encryption (PHE) and less computational overhead than Fully Homomorphic Encryption (FHE), we consider a Reinforcement Learning (RL) architecture over Leveled Homomorphic Encryption (LHE). We first show that the impact of the encryption noise under the Cheon-Kim-Kim-Song (CKKS) encryption scheme on the convergence of the model-based tabular Value Iteration (VI) can be analytically bounded. We also consider secure implementations of TD(0), SARSA(0) and Z-learning algorithms over the CKKS scheme, where we numerically demonstrate that the effects of the encryption noise on these algorithms are also minimal.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑