LocalValueBench：一个协作构建且可扩展的基准，用于评估大型语言模型中的本地化价值对齐与伦理安全
LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models
AI总结：
本文提出可扩展基准 LocalValueBench，通过伦理推理类型学、质询与提示工程评估 LLM 的澳大利亚本地价值对齐，并比较三款美国商业模型，为监管机构构建本地化基准提供工具。
摘要：
大型语言模型（LLM）的普及要求对其与本地价值观和伦理标准的一致性进行稳健评估，尤其是现有基准往往反映其创建者的文化、法律和意识形态价值观。本文提出的 LocalValueBench 是一个可扩展基准，旨在评估 LLM 对澳大利亚价值观的遵循情况，并为全球监管机构开发各自的本地价值对齐 LLM 基准提供框架。我们采用一种新颖的伦理推理类型学和质询方法，策划了全面的问题，并利用提示工程策略探测 LLM 的价值对齐情况。我们的评估标准量化了与本地价值观的偏差，确保评估过程严谨。对美国供应商的三款商业 LLM 进行的比较分析揭示了关于其有效性和局限性的重要洞见，证明了价值对齐的关键重要性。本研究为监管机构创建定制基准提供了有价值的工具和方法，并强调了未来加强伦理 AI 发展的研究途径。
英文摘要：
The proliferation of large language models (LLMs) requires robust evaluation of their alignment with local values and ethical standards, especially as existing benchmarks often reflect the cultural, legal, and ideological values of their creators. \textsc{LocalValueBench}, introduced in this paper, is an extensible benchmark designed to assess LLMs' adherence to Australian values, and provides a framework for regulators worldwide to develop their own LLM benchmarks for local value alignment. Employing a novel typology for ethical reasoning and an interrogation approach, we curated comprehensive questions and utilized prompt engineering strategies to probe LLMs' value alignment. Our evaluation criteria quantified deviations from local values, ensuring a rigorous assessment process. Comparative analysis of three commercial LLMs by USA vendors revealed significant insights into their effectiveness and limitations, demonstrating the critical importance of value alignment. This study offers valuable tools and methodologies for regulators to create tailored benchmarks, highlighting avenues for future research to enhance ethical AI development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
