arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.12150cs.CLcs.AI
收藏

你的大语言模型其实是公平性支持者,你应该像这样提示它

Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like One

Nanyang Technological University(南洋理工大学);Sea AI Lab(Sea AI实验室);Xi’an Jiaotong University(西安交通大学);Centre for Frontier AI Research (CFAR), A*STAR(新加坡科技研究局前沿人工智能研究中心)

Tianlin Li, Xiaoyu Zhang, Chao Du, Tianyu Pang, Qian Liu, Qing Guo, Chao Shen, Yang Liu

AI总结:

针对LLMs在观点表达中偏向多数群体而忽视少数群体的问题,提出FairThinking流水线,通过自动生成特定角色提示LLMs以表达多样观点,并在多个模型上验证了其优越性能。

摘要:

大语言模型(LLMs)的广泛采用凸显了确保其公平性的迫切需求。然而,LLMs 经常呈现主导观点,同时忽略来自少数群体的替代观点,导致潜在偏见。我们假设这些违反公平的行为发生是因为 LLMs 使用代表大多数训练数据的人类个性来表达其观点。作为对此的回应,我们验证用特定角色提示 LLMs 可以允许 LLMs 表达多样的观点。基于这一洞察和观察,我们开发了 FairThinking,一种旨在自动生成角色以使 LLMs 能够为公平表达阐明多样观点的流水线。为了评估 FairThinking,我们创建了一个包含千项涵盖三个与公平相关主题的数据集,并在 GPT-3.5、GPT-4、Llama2 和 Mistral 上进行实验以证明其卓越的性能。

英文摘要:

The widespread adoption of large language models (LLMs) underscores the urgent need to ensure their fairness. However, LLMs frequently present dominant viewpoints while ignoring alternative perspectives from minority parties, resulting in potential biases. We hypothesize that these fairness-violating behaviors occur because LLMs express their viewpoints using a human personality that represents the majority of training data. In response to this, we validate that prompting LLMs with specific roles can allow LLMs to express diverse viewpoints. Building on this insight and observation, we develop FairThinking, a pipeline designed to automatically generate roles that enable LLMs to articulate diverse perspectives for fair expressions. To evaluate FairThinking, we create a dataset with a thousand items covering three fairness-related topics and conduct experiments on GPT-3.5, GPT-4, Llama2, and Mistral to demonstrate its superior performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑