LLM能否为多元人群发声？通过辩论微调LLM以生成可控的争议性陈述
Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
AI总结：
本文提出DEBATUNE流程，通过让持相反立场的LLMs进行多轮辩论生成高质量训练数据并微调模型，显著提升了LLMs生成可控争议性陈述的能力，且可泛化到新主题。
摘要：
让大语言模型（LLMs）为不同群体、尤其是少数群体发声，并生成支持其多元甚至争议性观点的陈述，对于营造包容性环境至关重要。然而，现有LLMs对其生成内容的立场缺乏足够的可控性，生成内容往往包含不一致、中立或有偏见的陈述。本文旨在提升LLMs在生成支持用户于提示中指定论点的陈述时的可控性。我们发现，两个持相反立场的LLMs之间进行多轮辩论，能够为每一方生成质量更高、观点更突出的陈述，而这些陈述是提升LLMs可控性的重要训练数据。受此启发，我们提出了一种新颖的辩论与微调（DEBATUNE）流程，通过微调LLMs来生成经辩论获得的陈述。为检验DEBATUNE，我们构建了迄今为止规模最大的辩论主题数据集，涵盖710个争议性主题及每个主题对应的论点。由GPT-4裁判采用新颖的争议可控性指标进行的评估表明，DEBATUNE显著提升了LLMs生成多元视角的能力。此外，这种可控性可泛化到未见过的主题，生成支持争议性论点的高质量陈述。
英文摘要：
Making LLMs speak for different, especially minority groups of people, and generate statements supporting their diverse or even controversial perspectives is critical to creating an inclusive environment. However, existing LLMs lack sufficient controllability to the stance of their generated content, which often contains inconsistent, neutral, or biased statements. In this paper, we improve the controllability of LLMs in generating statements supporting an argument the user defined in the prompt. We find that multi-round debates between two LLMs with opposite stances generate higher-quality and more salient statements for each, which are important training data to improve the controllability of LLMs. Motivated by this, we develop a novel debate & tuning (DEBATUNE) pipeline finetuning LLMs to generate the statements obtained via debate. To examine DEBATUNE, we curate the largest dataset of debate topics so far, which covers 710 controversial topics and corresponding arguments for each topic. Evaluations by the GPT-4 judge with a novel controversy controllability metric show that LLMs' capability of generating diverse perspectives is significantly improved by DEBATUNE. Moreover, such controllability can be generalized to unseen topics, generating high-quality statements supporting controversial arguments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
