For an artificial intelligence (AI) to be aligned with human values (or human preferences), it must first learn those values. AI systems that are trained on human behavior, risk miscategorising human irrationalities as human values -- and then optimising for these irrationalities. Simply learning human values still carries risks: AI learning them will inevitably also gain information on human irrationalities and human behaviour/policy. Both of these can be dangerous: knowing human policy allows an AI to become generically more powerful (whether it is partially aligned or not aligned at all), while learning human irrationalities allows it to exploit humans without needing to provide value in return. This paper analyses the danger in developing artificial intelligence that learns about human irrationalities and human policy, and constructs a model recommendation system with various levels of information about human biases, human policy, and human values. It concludes that, whatever the power and knowledge of the AI, it is more dangerous for it to know human irrationalities than human values. Thus it is better for the AI to learn human values directly, rather than learning human biases and then deducing values from behaviour.
Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization
Centre for Perceptual and Interactive Intelligence(感知与交互智能研究中心);The Chinese University of Hong Kong(香港中文大学);University of California, Los Angeles(加州大学洛杉矶分校)
ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspective
Institute of Data Science and Artificial Intelligence (DATAI), Universidad de Navarra(纳瓦拉大学数据科学与人工智能研究所);Information Systems Department, University Center for Economic and Administrative Sciences (CUCEA), University of Guadalajara(瓜达拉哈拉大学经济与行政科学中心信息系统系);Quantitative Methods Department, Universidad Pontificia Comillas(康普顿斯大学定量方法系)