创造力已离开聊天：语言模型去偏的代价
Creativity Has Left the Chat: The Price of Debiasing Language Models
AI总结：
本文通过 Llama-2 系列实验发现 RLHF 对齐会降低语言模型的句法和语义多样性，使其趋向低熵和吸引子状态，揭示了去偏与创造力之间的权衡，并强调提示工程对挖掘基础模型创造潜力的作用。
摘要：
大型语言模型（LLMs）已经彻底改变了自然语言处理，但可能表现出偏见并生成有毒内容。虽然诸如基于人类反馈的强化学习（RLHF）等对齐技术减少了这些问题，但它们对创造力的影响——创造力被定义为句法和语义多样性——仍未得到探索。我们通过三个聚焦于 Llama-2 系列的实验，研究了 RLHF 对 LLMs 创造力的非预期后果。我们的发现表明，对齐后的模型在 token 预测中表现出更低的熵，在嵌入空间中形成不同的簇，并趋向于“吸引子状态”，这表明输出多样性有限。我们的发现对依赖 LLMs 进行创意任务（如文案撰写、广告创作和客户画像生成）的营销人员具有重要意义。在为特定应用选择合适模型时，应仔细考虑对齐模型中一致性与创造力之间的权衡。我们还讨论了提示工程在挖掘基础模型创造潜力方面的重要性。
英文摘要：
Large Language Models (LLMs) have revolutionized natural language processing but can exhibit biases and may generate toxic content. While alignment techniques like Reinforcement Learning from Human Feedback (RLHF) reduce these issues, their impact on creativity, defined as syntactic and semantic diversity, remains unexplored. We investigate the unintended consequences of RLHF on the creativity of LLMs through three experiments focusing on the Llama-2 series. Our findings reveal that aligned models exhibit lower entropy in token predictions, form distinct clusters in the embedding space, and gravitate towards "attractor states", indicating limited output diversity. Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation. The trade-off between consistency and creativity in aligned models should be carefully considered when selecting the appropriate model for a given application. We also discuss the importance of prompt engineering in harnessing the creative potential of base models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
