There is a growing focus on how to design safe artificial intelligent (AI) agents. As systems become more complex, poorly specified goals or control mechanisms may cause AI agents to engage in unwanted and harmful outcomes. Thus it is necessary to design AI agents that follow initial programming intentions as the program grows in complexity. How to specify these initial intentions has also been an obstacle to designing safe AI agents. Finally, there is a need for the AI agent to have redundant safety mechanisms to ensure that any programming errors do not cascade into major problems. Humans are autonomous intelligent agents that have avoided these problems and the present manuscript argues that by understanding human self-regulation and goal setting, we may be better able to design safe AI agents. Some general principles of human self-regulation are outlined and specific guidance for AI design is given.
Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation
Department of Computer Science, University of Verona（威尼斯大学计算机科学系）；Department of Engineering for Innovation Medicine, University of Verona（威尼斯大学创新医学工程系）；Massachusetts Institute of Technology（麻省理工学院）
SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoning
Beijing Institute of Technology（北京理工大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development
University of Stuttgart（斯图加特大学）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所（IPA））；Fraunhofer Institute for Industrial Engineering IAO（弗劳恩霍夫工业工程研究所（IAO））；AUDI AG（奥迪股份公司）
Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy
Yale University（耶鲁大学）；National Library of Medicine, National Institutes of Health（国家医学图书馆，国立卫生研究院）；Mila-Quebec AI Institute（魁北克AI研究所）；Shanghai Jiao Tong University（上海交通大学）；OPPO Research Institute（OPPO研究院）；Reichman University（里奇曼大学）
Auction-Based Regulation for Artificial Intelligence
Department of Computer Science, University of Maryland, College Park, MD, USA（马里兰大学计算机科学系，College Park, MD, USA）；SAP Labs, LLC（SAP实验室）；Department of Computer Science, University of Central Florida, FL, USA（佛罗里达大学计算机科学系，FL, USA）
