MOSAIC：揭示大型语言模型的道德、社会和个体维度
MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models
AI总结：
MOSAIC首次提出评估大型语言模型道德、社会和个体维度的综合基准测试，通过九个问卷和四个游戏全面考察伦理推理能力，揭示MFT的局限性。
摘要：
大型语言模型（LLMs）正在越来越多地应用于敏感领域，包括心理支持、医疗保健和高风险决策。这种扩展促使了对LLM行为背后伦理和道德基础的研究，提出了关于其在伦理推理中可靠性的重要问题。然而，现有研究和基准测试几乎完全依赖道德基础理论（MFT），忽视了其他相关维度，如社会价值观、个性特征和个体特征，这些因素塑造了人类的伦理推理。为了解决这些局限性，我们引入了MOSAIC，这是首个大规模基准测试，旨在共同评估LLM的道德、社会和个体特征。该基准测试包含九个经过验证的问卷，来源于道德哲学、心理学和社会理论，以及四个基于平台的游戏，用于探测道德模糊场景。总共，MOSAIC包含超过600个精心挑选的问题和场景，作为可直接使用的、可扩展的资源，用于评估LLM的行为基础。我们验证了该基准测试在三个不同家族的模型上的有效性，证明了其在所有评估维度上的实用性，并提供了首次实证证据，即MFT单独不足以全面评估复杂人工智能系统的伦理行为。我们公开发布了数据集和我们的基准Python库。
英文摘要：
Large Language Models (LLMs) are increasingly deployed in sensitive applications including psychological support, healthcare, and high-stakes decision-making. This expansion has motivated growing research into the ethical and moral foundations underlying LLM behavior, raising critical questions about their reliability in ethical reasoning. However, existing studies and benchmarks rely almost exclusively on Moral Foundation Theory (MFT), largely neglecting other relevant dimensions such as social values, personality traits, and individual characteristics that shape human ethical reasoning. To address these limitations, we introduce MOSAIC, the first large-scale benchmark designed to jointly assess the moral, social, and individual characteristics of LLMs. The benchmark comprises nine validated questionnaires drawn from moral philosophy, psychology, and social theory, alongside four platform-based games designed to probe morally ambiguous scenarios. In total, MOSAIC includes over 600 curated questions and scenarios, released as a ready-to-use, extensible resource for evaluating the behavioral foundations of LLMs. We validate the benchmark across three models from different families, demonstrating its utility across all assessed dimensions and providing the first empirical evidence that MFT alone is insufficient to comprehensively evaluate complex AI systems' ethical behavior. We publicly release the dataset and our benchmark Python library.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
