arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.10165cs.LGcs.AIcs.CRcs.SYeess.SY
收藏

隐私保护强化学习超出预期

Privacy-Preserving Reinforcement Learning Beyond Expectation

University of Washington(华盛顿大学);Western Washington University(西华盛顿大学)

Arezoo Rajabi, Bhaskar Ramasubramanian, Abdullah Al Maruf, Radha Poovendran

摘要:

配备了机器学习算法的网络和网络物理系统(如自动驾驶汽车)与人类共享环境。在这样的设置中,将系统(或代理)行为与一个或多个人类用户的偏好对齐是很重要的。我们考虑的情况下,代理人必须在一个未知的环境中学习行为。我们的目标是捕捉人类的两个定义特征:i)评估和量化风险的倾向,以及ii)保持决策对外部各方隐藏的愿望。我们将累积前景理论(CPT)的强化学习(RL)的问题,前者的目标。对于后者,我们使用差分隐私。我们设计了一种算法,使RL代理能够学习策略,以保护隐私的方式最大化基于CPT的目标,并在奖励足够接近时对算法学习的价值函数的隐私建立保证。这是通过在每一步使用高斯过程机制添加校准噪声来实现的。通过实证评估,我们强调了隐私-效用权衡,并证明RL代理能够以隐私保护的方式学习与同一环境中的人类用户一致的行为

英文摘要:

Cyber and cyber-physical systems equipped with machine learning algorithms such as autonomous cars share environments with humans. In such a setting, it is important to align system (or agent) behaviors with the preferences of one or more human users. We consider the case when an agent has to learn behaviors in an unknown environment. Our goal is to capture two defining characteristics of humans: i) a tendency to assess and quantify risk, and ii) a desire to keep decision making hidden from external parties. We incorporate cumulative prospect theory (CPT) into the objective of a reinforcement learning (RL) problem for the former. For the latter, we use differential privacy. We design an algorithm to enable an RL agent to learn policies to maximize a CPT-based objective in a privacy-preserving manner and establish guarantees on the privacy of value functions learned by the algorithm when rewards are sufficiently close. This is accomplished through adding a calibrated noise using a Gaussian process mechanism at each step. Through empirical evaluations, we highlight a privacy-utility tradeoff and demonstrate that the RL agent is able to learn behaviors that are aligned with that of a human user in the same environment in a privacy-preserving manner

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑