arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.08061cs.CL
收藏

再三考虑,我们还是不要逐步思考了!零样本推理中的偏见与毒性

On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning

Stanford University(斯坦福大学);Shanghai Jiao Tong University(上海交通大学);Georgia Institute of Technology(佐治亚理工学院)

Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, Diyi Yang

AI总结:

本文通过在有害问题与刻板印象基准上的受控评估,发现零样本 CoT 会显著增加 LLM 产生有害输出的风险,并受模型规模和指令遵循能力影响。

摘要:

生成思维链(Chain of Thought, CoT)已被证明能够持续提升大型语言模型(LLM)在广泛 NLP 任务上的表现。然而,先前工作主要集中于逻辑推理任务(例如算术、常识问答);对于更多样化的推理类型,尤其是在社会情境中,这种提升是否仍然成立尚不清楚。具体而言,我们在两个社会敏感领域对零样本 CoT 进行了受控评估:有害问题和刻板印象基准。我们发现,在敏感领域中,零样本 CoT 推理会显著提高模型生成有害或不良输出的可能性,而且这一趋势在不同提示格式和模型变体中均成立。此外,我们表明有害 CoT 会随模型规模增大而增加,但会随着指令遵循能力的提升而减少。我们的工作表明,在具有社会重要性的任务上应谨慎使用零样本 CoT,尤其是在涉及边缘化群体或敏感话题时。

英文摘要:

Generating a Chain of Thought (CoT) has been shown to consistently improve large language model (LLM) performance on a wide range of NLP tasks. However, prior work has mainly focused on logical reasoning tasks (e.g. arithmetic, commonsense QA); it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts. Concretely, we perform a controlled evaluation of zero-shot CoT across two socially sensitive domains: harmful questions and stereotype benchmarks. We find that zero-shot CoT reasoning in sensitive domains significantly increases a model's likelihood to produce harmful or undesirable output, with trends holding across different prompt formats and model variants. Furthermore, we show that harmful CoTs increase with model size, but decrease with improved instruction following. Our work suggests that zero-shot CoT should be used with caution on socially important tasks, especially when marginalized groups or sensitive topics are involved.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑