arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.09799cs.CL
收藏

我们能从大型语言模型中提取多样化观点到何种程度?

How Far Can We Extract Diverse Perspectives from Large Language Models?

University of Minnesota(明尼苏达大学);Google DeepMind(谷歌DeepMind)

Shirley Anugrah Hayati, Minhwa Lee, Dheeraj Rajagopal, Dongyeop Kang

AI总结:

本研究探索大型语言模型在主观主题上生成多样化观点的能力,提出基于标准的提示和逐步召回提示技术,发现其提取最大多样性的表现与人类相当。

摘要:

收集多样化的人类意见成本高昂且具有挑战性。这导致了近期利用大型语言模型(LLMs)生成多样化数据以寻求潜在可扩展和高效解决方案的趋势。然而,LLMs在多大程度上能够针对主观主题生成多样化观点仍不清楚。在本研究中,我们探索了LLMs在主观主题(如社会规范和论证性文本)上生成多样化观点和理由的能力。我们引入了从LLMs中提取最大多样性的问题。受人类基于价值观形成观点的启发,我们提出了一种基于标准的提示技术来为多样化意见提供基础。为了探究我们能从LLMs中提取多样化观点到何种程度,即多样性覆盖率,我们采用逐步召回提示来迭代地从模型生成更多输出。我们的方法应用于各种任务,结果表明LLMs确实能够根据任务主观性程度产生多样化意见。我们还发现,LLM提取最大多样性的表现与人类相当。

英文摘要:

Collecting diverse human opinions is costly and challenging. This leads to a recent trend in exploiting large language models (LLMs) for generating diverse data for potential scalable and efficient solutions. However, the extent to which LLMs can generate diverse perspectives on subjective topics is still unclear. In this study, we explore LLMs' capacity of generating diverse perspectives and rationales on subjective topics such as social norms and argumentative texts. We introduce the problem of extracting maximum diversity from LLMs. Motivated by how humans form opinions based on values, we propose a criteria-based prompting technique to ground diverse opinions. To see how far we can extract diverse perspectives from LLMs, or called diversity coverage, we employ a step-by-step recall prompting to generate more outputs from the model iteratively. Our methods, applied to various tasks, show that LLMs can indeed produce diverse opinions according to the degree of task subjectivity. We also find that LLM's performance of extracting maximum diversity is on par with human.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑