多模态分析的因果交叉性与梯度对偶形式：以仇恨模因为例的案例研究
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes
AI总结：
本文以仇恨模因为例，结合因果分析与可解释AI，将交叉性构建为平均处理效应（ATE），揭示Transformer模型行为差异及Llama-2通过上下文学习识别交叉性的机制，并利用元梯度解释学习过程。
摘要：
随着机器学习（ML）和大型语言模型（LLMs）的快速扩展，理解其机制内的语义至关重要。因果分析定义了语义，而基于梯度的方法对于可解释AI（XAI）至关重要，用于解释模型的“黑盒”。结合这两者，我们研究了模型的机制如何揭示其对基于证据的决策的因果效应。研究表明，交叉性——即个体人口统计特征的综合影响——可以被构建为平均处理效应（ATE）。本文证明，仇恨模因检测可以利用交叉性原则被视为一种ATE估计，并且基于梯度的注意力得分总结突出了三个Transformer模型的不同行为。我们进一步揭示，LLM Llama-2能够通过上下文学习识别检测的交叉性方面，并且该学习过程可以通过元梯度（一种梯度的二次形式）来解释。总之，这项工作推进了关于因果性和XAI的对话。我们的代码可在线获取（见外部资源部分）。
英文摘要：
Amidst the rapid expansion of Machine Learning (ML) and Large Language Models (LLMs), understanding the semantics within their mechanisms is vital. Causal analyses define semantics, while gradient-based methods are essential to eXplainable AI (XAI), interpreting the model's 'black box'. Integrating these, we investigate how a model's mechanisms reveal its causal effect on evidence-based decision-making. Research indicates intersectionality - the combined impact of an individual's demographics - can be framed as an Average Treatment Effect (ATE). This paper demonstrates that hateful meme detection can be viewed as an ATE estimation using intersectionality principles, and summarized gradient-based attention scores highlight distinct behaviors of three Transformer models. We further reveal that LLM Llama-2 can discern the intersectional aspects of the detection through in-context learning and that the learning process could be explained via meta-gradient, a secondary form of gradient. In conclusion, this work furthers the dialogue on Causality and XAI. Our code is available online (see External Resources section).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
