微调大语言模型是追踪书籍中社会偏见演变的“时间胶囊”
Fine-Tuned LLMs are "Time Capsules" for Tracking Societal Bias Through Books
AI总结：
针对书籍反映的时代社会偏见被大语言模型（LLMs）学习和延续的问题，该研究构建BookPAGE语料库，通过在不同年代书籍上微调LLMs并分析其偏见变化，揭示了性别、种族等偏见的时代趋势，证明偏见源于书籍内容，为追踪社会偏见提供新方法。
摘要：
书籍虽常蕴含丰富文化洞见，但也可能反映其时代的社会偏见——大语言模型（LLMs）在训练过程中可能学习并延续这些偏见。我们提出一种利用微调LLMs追踪和量化这些偏见的新方法。我们构建了BookPAGE语料库，包含1950-2019年七个十年间的593本虚构书籍，以追踪偏见演变。通过在每个十年的书籍上微调LLMs并使用针对性提示，我们研究了性别、性取向、种族和宗教相关偏见的变化。研究结果表明，在特定十年书籍上训练的LLMs会表现出其时代的偏见，既有渐进趋势也有显著转变。例如，模型回应显示1950年代至2010年代女性领导角色描绘比例从8%逐步提升至22%，1990年代显著上升（从4%升至12%），可能与第三波女权主义相符；1980年代至2000年代同性关系提及率从0%大幅增至10%，反映LGBTQ+可见度提升；令人担忧的是，2000年代对伊斯兰教的负面描绘从26%骤升至38%，可能反映9·11事件后的情绪。重要的是，我们证明这些偏见主要源于书籍内容，而非模型架构或初始训练。我们的研究通过连接AI、文学研究和社会科学，为社会偏见趋势提供了新视角。
英文摘要：
Books, while often rich in cultural insights, can also mirror societal biases of their eras - biases that Large Language Models (LLMs) may learn and perpetuate during training. We introduce a novel method to trace and quantify these biases using fine-tuned LLMs. We develop BookPAGE, a corpus comprising 593 fictional books across seven decades (1950-2019), to track bias evolution. By fine-tuning LLMs on books from each decade and using targeted prompts, we examine shifts in biases related to gender, sexual orientation, race, and religion. Our findings indicate that LLMs trained on decade-specific books manifest biases reflective of their times, with both gradual trends and notable shifts. For example, model responses showed a progressive increase in the portrayal of women in leadership roles (from 8% to 22%) from the 1950s to 2010s, with a significant uptick in the 1990s (from 4% to 12%), possibly aligning with third-wave feminism. Same-sex relationship references increased markedly from the 1980s to 2000s (from 0% to 10%), mirroring growing LGBTQ+ visibility. Concerningly, negative portrayals of Islam rose sharply in the 2000s (26% to 38%), likely reflecting post-9/11 sentiments. Importantly, we demonstrate that these biases stem mainly from the books' content and not the models' architecture or initial training. Our study offers a new perspective on societal bias trends by bridging AI, literary studies, and social science research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
