探究大语言模型的因果性操纵
Probing Causality Manipulation of Large Language Models
AI总结：
本文提出一种分层探究大语言模型因果性操纵的新方法，利用RAG和ICL在因果分类任务上实验，发现LLMs能识别直接因果关系但缺乏专门认知。
摘要：
大语言模型（LLMs）在自然语言处理中展示了各种能力，包括关于因果性的问题。让LLMs掌握因果性并不直观，因为预训练模型通常基于统计关联工作，并不关注句子中的原因和结果。因此，探究因果性的内部操纵对LLMs是必要的。本文提出一种新方法，通过向模型提供不同捷径并观察行为，分层探究因果性操纵。我们利用检索增强生成（RAG）和上下文学习（ICL），在设计的因果性分类任务上对模型进行实验。我们在主流LLMs上开展实验，包括GPT-4及一些更小和特定领域的模型。结果表明，LLMs能检测与因果性相关的实体并识别直接因果关系。然而，LLMs缺乏对因果性的专门认知，仅将其作为句子全局语义的一部分来处理。
英文摘要：
Large language models (LLMs) have shown various ability on natural language processing, including problems about causality. It is not intuitive for LLMs to command causality, since pretrained models usually work on statistical associations, and do not focus on causes and effects in sentences. So that probing internal manipulation of causality is necessary for LLMs. This paper proposes a novel approach to probe causality manipulation hierarchically, by providing different shortcuts to models and observe behaviors. We exploit retrieval augmented generation (RAG) and in-context learning (ICL) for models on a designed causality classification task. We conduct experiments on mainstream LLMs, including GPT-4 and some smaller and domain-specific models. Our results suggest that LLMs can detect entities related to causality and recognize direct causal relationships. However, LLMs lack specialized cognition for causality, merely treating them as part of the global semantic of the sentence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
