arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.08698cs.CLcs.CY
收藏

SocialGaze:改进大型语言模型对人类社会规范的整合

SocialGaze: Improving the Integration of Human Social Norms in Large Language Models

UNC Chapel Hill(北卡罗来纳大学教堂山分校)

Anvesh Rao Vijjini, Rakesh R. Menon, Jiayi Fu, Shashank Srivastava, Snigdha Chaturvedi

AI总结:

针对LLMs与社会规范不一致的问题,提出多步提示框架SocialGaze,通过多视角言语化社会情境来提升社会接受度判断与人类共识的契合度,并揭示了模型在归咎责任时与性别和年龄相关的偏见。

摘要:

尽管近年来大量研究探索了增强大型语言模型(LLMs)的推理能力,但在理解这些模型与社会价值观和规范的契合度方面仍存在空白。我们引入了判断社会接受度的任务。社会接受度要求模型判断并合理化人们在社会情境中行为的可接受性。例如,邻居要求社区其他人晚上将宠物留在室内,这在社会上是否可接受?我们发现,LLMs对社会接受度的理解常常与人类共识不一致。为缓解此问题,我们引入了SocialGaze,这是一个多步提示框架,语言模型在形成判断前,会从多个视角将社会情境言语化。实验表明,SocialGaze方法在GPT-3.5模型上将与人类判断的契合度提高了11个F1点。我们还发现了LLMs在归咎责任时与性别(男性更可能被不公平评判)和年龄(对于年长叙述者,LLMs与人类更契合)等特征相关的偏见和相关性。

英文摘要:

While much research has explored enhancing the reasoning capabilities of large language models (LLMs) in the last few years, there is a gap in understanding the alignment of these models with social values and norms. We introduce the task of judging social acceptance. Social acceptance requires models to judge and rationalize the acceptability of people's actions in social situations. For example, is it socially acceptable for a neighbor to ask others in the community to keep their pets indoors at night? We find that LLMs' understanding of social acceptance is often misaligned with human consensus. To alleviate this, we introduce SocialGaze, a multi-step prompting framework, in which a language model verbalizes a social situation from multiple perspectives before forming a judgment. Our experiments demonstrate that the SocialGaze approach improves the alignment with human judgments by up to 11 F1 points with the GPT-3.5 model. We also identify biases and correlations in LLMs in assigning blame that is related to features such as the gender (males are significantly more likely to be judged unfairly) and age (LLMs are more aligned with humans for older narrators).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑