通过提示知识微调去偏大型语言模型在在线行为分析中的社会因素
Debiasing Large Language Models toward Social Factors in Online Behavior Analytics through Prompt Knowledge Tuning
AI总结：
本文研究了在社交情境中通过引入用户目标推断 dispositional 因果性和消息上下文推断 situational 因果性对 LLM 性能的影响，提出了一种通过增强指令提示来减少社会归因偏见的方法，提升零样本分类任务的性能。
摘要：
归因理论解释了个体如何在社会情境中通过个人（ dispositional ）和非个人（ situational ）因果性来解释和归因他人的行为。训练于人类生成语料库的大型语言模型（LLMs）可能在社会情境中隐含地模仿这种社会归因过程。然而，LLMs 在推理中利用这些因果归因的程度仍不明确。尽管使用推理范式，如链式推理（CoT），在各种任务中显示出有希望的结果，但在推理中忽略社会归因可能导致 LLMs 在社会情境中产生偏见响应。在本研究中，我们探讨了在推理中整合用户目标作为知识推断 dispositional 因果性以及消息上下文推断 situational 因果性对 LLM 性能的影响。为此，我们引入了一种可扩展的方法，通过在 LLM 的指令提示中加入两种提示辅助，基于社交信息的上下文和目标，以缓解此类偏见。该方法在零样本分类任务中提高了模型性能，同时减少了 LLM 在行为分析应用中的社会归因偏见。我们实证地展示了在灾难领域社交媒体的意图检测和主题检测任务中，考虑灾难类型的变化和社交媒体的多种语言时，我们方法的益处。我们的实验突显了三个开源 LLMs（Llama3、Mistral 和 Gemma）对社会归因的偏见，并展示了我们缓解策略的有效性。
英文摘要：
Attribution theory explains how individuals interpret and attribute others' behavior in a social context by employing personal (dispositional) and impersonal (situational) causality. Large Language Models (LLMs), trained on human-generated corpora, may implicitly mimic this social attribution process in social contexts. However, the extent to which LLMs utilize these causal attributions in their reasoning remains underexplored. Although using reasoning paradigms, such as Chain-of-Thought (CoT), has shown promising results in various tasks, ignoring social attribution in reasoning could lead to biased responses by LLMs in social contexts. In this study, we investigate the impact of incorporating a user's goal as knowledge to infer dispositional causality and message context to infer situational causality on LLM performance. To this end, we introduce a scalable method to mitigate such biases by enriching the instruction prompts for LLMs with two prompt aids using social-attribution knowledge, based on the context and goal of a social media message. This method improves the model performance while reducing the social-attribution bias of the LLM in the reasoning on zero-shot classification tasks for behavior analytics applications. We empirically show the benefits of our method across two tasks-intent detection and theme detection on social media in the disaster domain-when considering the variability of disaster types and multiple languages of social media. Our experiments highlight the biases of three open-source LLMs: Llama3, Mistral, and Gemma, toward social attribution, and show the effectiveness of our mitigation strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
