arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.13985cs.AI
收藏

算法学习人类价值观和非理性的危险

The dangers in algorithms learning humans' values and irrationalities

Berkeley Existential Risk Initiative(伯克利存在风险倡议组织);Aligned AI;Future of Humanity Institute(人类未来研究所);Oxford University(牛津大学);Machine Intelligence Research Institute(机器智能研究所)

Rebecca Gorman, Stuart Armstrong

摘要:

为了让人工智能(AI)与人类价值观(或人类偏好)保持一致,它必须首先学习这些价值观。根据人类行为训练的人工智能系统,有可能将人类的非理性错误归类为人类价值观,然后对这些非理性进行优化。简单地学习人类价值观仍然存在风险:人工智能学习它们也将不可避免地获得有关人类非理性和人类行为/政策的信息。这两种情况都可能是危险的:了解人类的政策可以让人工智能变得更强大(无论它是部分对齐还是根本不对齐),而学习人类的非理性可以让它利用人类而不需要提供价值作为回报。本文分析了发展人工智能的危险性,学习人类的非理性和人类的政策,并构建了一个模型推荐系统与不同层次的信息,人类的偏见,人类的政策,和人类的价值观。它的结论是,无论人工智能的力量和知识如何,它知道人类的非理性比人类的价值观更危险。因此,人工智能最好直接学习人类的价值观,而不是学习人类的偏见,然后从行为中推导出价值观。

英文摘要:

For an artificial intelligence (AI) to be aligned with human values (or human preferences), it must first learn those values. AI systems that are trained on human behavior, risk miscategorising human irrationalities as human values -- and then optimising for these irrationalities. Simply learning human values still carries risks: AI learning them will inevitably also gain information on human irrationalities and human behaviour/policy. Both of these can be dangerous: knowing human policy allows an AI to become generically more powerful (whether it is partially aligned or not aligned at all), while learning human irrationalities allows it to exploit humans without needing to provide value in return. This paper analyses the danger in developing artificial intelligence that learns about human irrationalities and human policy, and constructs a model recommendation system with various levels of information about human biases, human policy, and human values. It concludes that, whatever the power and knowledge of the AI, it is more dangerous for it to know human irrationalities than human values. Thus it is better for the AI to learn human values directly, rather than learning human biases and then deducing values from behaviour.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑