arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.14929cs.CL
收藏

将语言模型与用户意见对齐

Aligning Language Models to User Opinions

UC San Diego(加州大学圣地亚哥分校);Allen Institute of AI(艾伦人工智能研究所);University of British Columbia(不列颠哥伦比亚大学);Vector Institute for AI(矢量人工智能研究所)

EunJeong Hwang, Bodhisattwa Prasad Majumder, Niket Tandon

AI总结:

本文研究如何将大语言模型与特定用户意见对齐,通过同时建模用户意见、人口统计和意识形态,并利用用户过去的相关意见,在预测公众意见时准确率提升了7个百分点。

摘要:

开发与人类交互的LLM的一个重要方面是使模型的行为与其用户对齐。可以通过提示让LLM表现出特定人设,尤其是模型在预训练阶段捕获的用户群体或意识形态人设。但是,如何最好地将LLM与特定用户对齐,而不是与人口统计或意识形态群体对齐,仍然是一个悬而未决的问题。通过挖掘公众意见调查(由Pew Research进行),我们发现用户的意见与其人口统计和意识形态并不是互为预测因子。我们利用这一见解来对齐LLM,同时对用户意见以及用户人口统计和意识形态进行建模,在预测涵盖广泛主题的调查问题中的公众意见时,实现了高达7个点的准确率提升。除了使用人口统计和意识形态提示LLM的典型方法外,我们还发现利用个别用户最相关的过去意见使模型能够更准确地预测用户意见。

英文摘要:

An important aspect of developing LLMs that interact with humans is to align models' behavior to their users. It is possible to prompt an LLM into behaving as a certain persona, especially a user group or ideological persona the model captured during its pertaining stage. But, how to best align an LLM with a specific user and not a demographic or ideological group remains an open question. Mining public opinion surveys (by Pew Research), we find that the opinions of a user and their demographics and ideologies are not mutual predictors. We use this insight to align LLMs by modeling both user opinions as well as user demographics and ideology, achieving up to 7 points accuracy gains in predicting public opinions from survey questions across a broad set of topics. In addition to the typical approach of prompting LLMs with demographics and ideology, we discover that utilizing the most relevant past opinions from individual users enables the model to predict user opinions more accurately.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑