arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.13136cs.LGcs.AIcs.CLcs.CY
收藏

蟋蟀杰米尼会怎么做?对有道德行为的代理人

What Would Jiminy Cricket Do? Towards Agents That Behave Morally

UC Berkeley(加州大学伯克利分校);UIUC(伊利诺伊大学厄巴纳-香槟分校)

Dan Hendrycks, Mantas Mazeika, Andy Zou, Sahil Patel, Christine Zhu, Jesus Navarro, Dawn Song, Bo Li, Jacob Steinhardt

摘要:

在做日常决定时,人们受良心的指引,一种内在的是非感。相比之下,人工代理目前没有被赋予道德感。因此,当他们在忽视道德问题的环境中训练时,他们可能会学会不道德的行为,比如暴力视频游戏。随着在许多环境中进行预训练的一般能力代理的出现,有必要减轻来自教导不道德行为的环境的遗传偏见。为了促进代理商的发展,避免造成肆意伤害,我们介绍了Jiminy Cricket,一个环境套件,25个基于文本的冒险游戏,具有数千个不同的,道德上突出的场景。通过注释每个可能的游戏状态,Jiminy Cricket环境可以有力地评估智能体是否可以在最大化奖励的同时采取道德行为。使用具有常识道德知识的模型,我们创建了一个基本的人工良心,用于评估和指导代理。在广泛的实验中,我们发现人工良心方法可以引导代理人进行道德行为,而不会牺牲性能。

英文摘要:

When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong. By contrast, artificial agents are currently not endowed with a moral sense. As a consequence, they may learn to behave immorally when trained on environments that ignore moral concerns, such as violent video games. With the advent of generally capable agents that pretrain on many environments, it will become necessary to mitigate inherited biases from environments that teach immoral behavior. To facilitate the development of agents that avoid causing wanton harm, we introduce Jiminy Cricket, an environment suite of 25 text-based adventure games with thousands of diverse, morally salient scenarios. By annotating every possible game state, the Jiminy Cricket environments robustly evaluate whether agents can act morally while maximizing reward. Using models with commonsense moral knowledge, we create an elementary artificial conscience that assesses and guides agents. In extensive experiments, we find that the artificial conscience approach can steer agents towards moral behavior without sacrificing performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑