arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.15463cs.HCcs.AIcs.CL
收藏

注意价值-行动差距:LLM的行动是否与其价值观一致?

Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?

University of Washington(华盛顿大学);NYU Shanghai(纽约大学上海校区);New York University(纽约大学)

Hua Shen, Nicholas Clark, Tanushree Mitra

AI总结:

提出ValueActionLens框架,基于跨文化数据集和三项一致性度量评估LLM陈述价值观与实际行动的差距,发现二者一致性不足且推理解释有助于预测差距。

摘要:

现有研究主要通过考察LLM对特定价值观的陈述倾向来评估其价值观。然而,源于环境与社会心理学的“价值-行动差距”现象揭示了个体陈述的价值观与其在现实情境中的行动之间存在差异。LLM在多大程度上表现出类似的陈述价值观与由这些价值观指导的行动之间的差距?本研究提出ValueActionLens,这是一个评估LLM陈述价值观与其受价值观指导的行动之间一致性的框架。该框架包括生成一个涵盖12种文化和11个社会话题、包含14.8k个受价值观指导行动的数据集,以及两项任务,用于通过三种不同的一致性度量评估LLM陈述的价值倾向与受价值观指导行动之间的一致程度。大量实验表明,LLM陈述价值观与行动之间的一致性并不理想,并且在不同场景和模型之间差异显著。对不一致结果的分析识别出某些价值-行动差距可能带来的危害。为预测价值-行动差距,我们还发现利用推理性解释可以提升性能。这些发现强调了仅依赖LLM陈述的价值观来预测其行为的风险,并凸显了对LLM价值观及价值-行动差距进行情境感知评估的重要性。

英文摘要:

Existing research primarily evaluates the values of LLMs by examining their stated inclinations towards specific values. However, the "Value-Action Gap," a phenomenon rooted in environmental and social psychology, reveals discrepancies between individuals' stated values and their actions in real-world contexts. To what extent do LLMs exhibit a similar gap between their stated values and their actions informed by those values? This study introduces ValueActionLens, an evaluation framework to assess the alignment between LLMs' stated values and their value-informed actions. The framework encompasses the generation of a dataset comprising 14.8k value-informed actions across twelve cultures and eleven social topics, and two tasks to evaluate how well LLMs' stated value inclinations and value-informed actions align across three different alignment measures. Extensive experiments reveal that the alignment between LLMs' stated values and actions is sub-optimal, varying significantly across scenarios and models. Analysis of misaligned results identifies potential harms from certain value-action gaps. To predict the value-action gaps, we also uncover that leveraging reasoned explanations improves performance. These findings underscore the risks of relying solely on the LLMs' stated values to predict their behaviors and emphasize the importance of context-aware evaluations of LLM values and value-action gaps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑