通过伦理困境对LLM进行亚里士多德式美德画像
Aristotelian Virtue Profiling of LLMs through Ethical Dilemmas
AI总结：
提出VirtueMap框架，通过七种非致命、非政治、非宗教的伦理困境，让人类或LLM对五种回应排序，基于95%共识的参考排序，使用归一化Borda对齐计算实践智慧、正义、诚实、勇气和节制的美德画像，在九个LLM家族中评估，平均排名一致性达90.3%。
摘要：
大型语言模型（LLM）经常面临伦理权衡，其中多种回应可能都是合理的，但表达了不同的优先级，如公平、诚实、勇气或克制。我们引入了VirtueMap，一个通过亚里士多德式美德伦理学视角描述这些模式的框架。VirtueMap不要求单一正确答案，而是要求人类或LLM对七个一般性、非致命、非政治、非宗教的伦理困境中的每一个，对五种回应进行排序。为了定义用于评分的参考排序，我们首先针对每个困境和美德，提出了五种回应从最到最不表达该美德的排序。然后，我们为每个排序收集了超过100个受访者评估，并仅在至少95%确认时才将其保留为操作性的真实标准。排名使用归一化Borda对齐对这些保留的排序进行评分，生成实践智慧、正义、诚实、勇气和克制的画像。我们将VirtueMap应用于九个LLM家族进行重复运行评估，发现平均排名一致性高（90.3%），最大的差异出现在勇气、克制和正义上。我们还发布了一个交互式网站，可在浏览器中本地计算画像，并将受访者与测量的LLM画像进行比较。
英文摘要：
Large Language Models (LLMs) often face ethical tradeoffs in which several responses may be defensible but express different priorities, such as fairness, honesty, courage, or restraint. We introduce VirtueMap, a framework for describing these patterns through an Aristotelian virtue-ethics lens. Instead of asking for a single correct answer, VirtueMap asks humans or LLMs to rank all five responses to each of seven general, non-lethal, non-political, and non-religious ethical dilemmas. To define the reference orderings used for scoring, we first proposed, for each dilemma and virtue, an ordering of the five responses from most to least expressive of that virtue. We then collected more than 100 respondent evaluations per ordering and retained it as operational ground truth only when at least 95% confirmed it. Rankings are scored against these retained orderings using normalized Borda alignment, yielding profiles over Practical Wisdom, Justice, Truthfulness, Courage, and Temperance. We apply VirtueMap to nine LLM families in a repeated-run evaluation and find high mean rank consistency (90.3%), with the largest differences appearing on Courage, Temperance, and Justice. We also release an interactive website that computes profiles locally in the browser and compares respondents with measured LLM profiles.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
