ValueBench:迈向全面评估大语言模型价值取向与价值理解
ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models
AI总结:
本文提出首个综合性心理测量基准ValueBench,通过44个量表、453个价值维度及真实交互流程,系统评估大语言模型的价值取向与价值理解,并在六个代表性模型上验证其接近专家结论的能力。
摘要:
大语言模型(LLMs)正在变革多个领域,并作为人类代理获得日益增长的影响力。这一发展凸显了评估LLMs价值取向与价值理解的迫切需求,以确保其负责任地融入面向公众的应用。本工作提出了ValueBench,这是首个用于评估LLMs价值取向和价值理解的综合性心理测量基准。ValueBench从44个既有心理测量量表中收集数据,涵盖453个多维价值维度。我们提出了一条基于真实人类-AI交互的评估流程,用于探测价值取向,并提出了在开放式价值空间中评估价值理解的新任务。通过在六个代表性LLMs上进行的大量实验,我们揭示了它们共有且独特的价值取向,并展示了它们在价值相关的抽取与生成任务中接近专家结论的能力。ValueBench可在https://github.com/Value4AI/ValueBench公开获取。
英文摘要:
Large Language Models (LLMs) are transforming diverse fields and gaining increasing influence as human proxies. This development underscores the urgent need for evaluating value orientations and understanding of LLMs to ensure their responsible integration into public-facing applications. This work introduces ValueBench, the first comprehensive psychometric benchmark for evaluating value orientations and value understanding in LLMs. ValueBench collects data from 44 established psychometric inventories, encompassing 453 multifaceted value dimensions. We propose an evaluation pipeline grounded in realistic human-AI interactions to probe value orientations, along with novel tasks for evaluating value understanding in an open-ended value space. With extensive experiments conducted on six representative LLMs, we unveil their shared and distinctive value orientations and exhibit their ability to approximate expert conclusions in value-related extraction and generation tasks. ValueBench is openly accessible at https://github.com/Value4AI/ValueBench.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。