arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.00813cs.AI
收藏

Goal Alignment:一个对价值对齐问题的人性化解释

Goal Alignment: A Human-Aware Account of Value Alignment Problem

Colorado State University(科罗拉多州立大学)

Malek Mechergui, Sarath Sreedharan

摘要:

当人工智能智能体的指定目标与其用户的真实潜在目标不匹配时,就会出现价值对齐问题。这个问题被广泛认为是人工智能的核心安全问题之一。不幸的是,大多数现有的作品在价值对齐往往集中在主要与奖励功能是一个不直观的机制,以指定目标的事实有关的问题。然而,目标指定机制的复杂性只是用户可能错误指定其目标的许多原因之一。这些作品忽视了错位的一个基本原因是人类对代理行为的预期和代理为特定目标所产生的行为的固有不对称。为了解决这个空白,我们提出了一个新的配方的价值对齐的问题,命名为目标对齐,专注于几个核心的挑战相关的价值对齐。在这样做的过程中,我们弥合了目前不同的研究领域的价值对齐和人类意识的规划。此外,我们提出了一个第一的同类交互式算法,能够使用不正确的信念下产生的信息代理,以确定真正的潜在目标的用户。

英文摘要:

Value alignment problems arise in scenarios where the specified objectives of an AI agent don't match the true underlying objective of its users. The problem has been widely argued to be one of the central safety problems in AI. Unfortunately, most existing works in value alignment tend to focus on issues that are primarily related to the fact that reward functions are an unintuitive mechanism to specify objectives. However, the complexity of the objective specification mechanism is just one of many reasons why the user may have misspecified their objective. A foundational cause for misalignment that is being overlooked by these works is the inherent asymmetry in human expectations about the agent's behavior and the behavior generated by the agent for the specified objective. To address this lacuna, we propose a novel formulation for the value alignment problem, named goal alignment that focuses on a few central challenges related to value alignment. In doing so, we bridge the currently disparate research areas of value alignment and human-aware planning. Additionally, we propose a first-of-its-kind interactive algorithm that is capable of using information generated under incorrect beliefs about the agent, to determine the true underlying goal of the user.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑