MoralityGym：用于评估序列决策代理中分层道德对齐的基准
MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents
AI总结：
本文提出MoralityGym基准，通过将道德规范表示为有序的规范约束，评估序列决策代理中分层道德对齐的挑战，展示了98个伦理困境问题，并通过心理学和哲学的见解改进了伦理决策方法。
摘要：
评估在面对冲突且分层结构的人类规范时，代理的道德对齐是一个在人工智能安全、道德哲学和认知科学交汇处的关键挑战。我们引入了Morality Chains，一种新的形式化方法，用于将道德规范表示为有序的规范约束，并引入了MoralityGym，一个包含98个伦理困境问题的基准，这些问题是作为电车困境风格的Gymnasium环境呈现的。通过将任务解决与道德评估解耦，并引入新的道德度量标准，MoralityGym允许将心理学和哲学的见解整合到规范敏感推理的评估中。基于安全强化学习方法的基准结果揭示了关键限制，强调了需要更系统的方法来处理伦理决策。本文为开发在复杂现实环境中行为更可靠、透明和道德的AI系统提供了基础。
英文摘要：
Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection of AI safety, moral philosophy, and cognitive science. We introduce Morality Chains, a novel formalism for representing moral norms as ordered deontic constraints, and MoralityGym, a benchmark of 98 ethical-dilemma problems presented as trolley-dilemma-style Gymnasium environments. By decoupling task-solving from moral evaluation and introducing a novel Morality Metric, MoralityGym allows the integration of insights from psychology and philosophy into the evaluation of norm-sensitive reasoning. Baseline results with Safe RL methods reveal key limitations, underscoring the need for more principled approaches to ethical decision-making. This work provides a foundation for developing AI systems that behave more reliably, transparently, and ethically in complex real-world contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
