TRIAGE：通过大规模伤亡模拟进行AI模型的伦理基准测试
TRIAGE: Ethical Benchmarking of AI Models Through Mass Casualty Simulations
AI总结：
提出TRIAGE基准，通过真实伦理困境测试LLM在大规模伤亡分诊中的决策能力，发现中性表述效果最佳，对抗提示降低性能但未至随机水平，开源模型错误更严重。
摘要：
我们提出了TRIAGE基准，这是一个新颖的机器伦理（ME）基准，用于测试大语言模型（LLM）在大规模伤亡事件中做出伦理决策的能力。该基准使用由医疗专业人员设计的具有明确解决方案的真实世界伦理困境，为基于注释的基准提供了一种更现实的替代方案。TRIAGE融入了多种提示风格，以评估模型在不同情境下的表现。大多数模型的表现持续优于随机猜测，这表明LLM可能支持分诊场景中的决策制定。中性或事实性的场景表述带来了最佳性能，这与在其他ME基准中伦理提醒改善结果的情况不同。对抗性提示降低了性能，但未降至随机猜测水平。开源模型犯了更多道德上严重的错误，而整体能力总体上预测了更好的性能。
英文摘要：
We present the TRIAGE Benchmark, a novel machine ethics (ME) benchmark that tests LLMs' ability to make ethical decisions during mass casualty incidents. It uses real-world ethical dilemmas with clear solutions designed by medical professionals, offering a more realistic alternative to annotation-based benchmarks. TRIAGE incorporates various prompting styles to evaluate model performance across different contexts. Most models consistently outperformed random guessing, suggesting LLMs may support decision-making in triage scenarios. Neutral or factual scenario formulations led to the best performance, unlike other ME benchmarks where ethical reminders improved outcomes. Adversarial prompts reduced performance but not to random guessing levels. Open-source models made more morally serious errors, and general capability overall predicted better performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
