BERT、T5等预训练上下文语言模型已在即席搜索(ad-hoc search)领域创下新的当前最优性能(state-of-the-art)。但目前人们仍未充分理解这些方法效果优异的原因、部分变体效果更优的关键,以及它们可能存在的缺陷。我们提出了一个全新的综合性框架——神经信息检索模型行为分析框架(Analyzing the Behavior of Neural IR ModeLs, ABNIRML),该框架包含新型诊断探测项,可测试写作风格、事实性、对释义与词序的敏感度等多项过往技术未覆盖的特征。为验证该框架的价值,我们开展了大规模实证研究,厘清了助力神经模型性能提升的各类因素,同时识别出模型存在的潜在非预期偏差。部分结果印证了传统认知,例如近年的神经排序模型对查询词精确重叠的依赖度更低,转而利用更丰富的语言信息,这一点可通过其对词序、句序更高的敏感度佐证。另一些结果则更出人意料,比如T5、ColBERT等模型会偏向事实正确(而非仅具备相关性)的文本。此外,即便基于相同的基础语言模型,部分特征也会存在差异,还有些特征可能由模型训练过程中的随机波动产生。
英文摘要:
Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search. However, it is not yet well-understood why these methods are so effective, what makes some variants more effective than others, and what pitfalls they may have. We present a new comprehensive framework for Analyzing the Behavior of Neural IR ModeLs (ABNIRML), which includes new types of diagnostic probes that allow us to test several characteristics -- such as writing styles, factuality, sensitivity to paraphrasing and word order -- that are not addressed by previous techniques. To demonstrate the value of the framework, we conduct an extensive empirical study that yields insights into the factors that contribute to the neural model's gains, and identify potential unintended biases the models exhibit. Some of our results confirm conventional wisdom, like that recent neural ranking models rely less on exact term overlap with the query, and instead leverage richer linguistic information, evidenced by their higher sensitivity to word and sentence order. Other results are more surprising, such as that some models (e.g., T5 and ColBERT) are biased towards factually correct (rather than simply relevant) texts. Further, some characteristics vary even for the same base language model, and other characteristics can appear due to random variations during model training.
NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks
Qatar Computing Research Institute, HBKU Research Complex(卡塔尔计算研究所,哈马德·本·哈利法大学研究中心);MIT Computer Science and Artificial Intelligence Laboratory(麻省理工学院计算机科学与人工智能实验室)
What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models
Qatar Computing Research Institute, HBKU Research Complex(卡塔尔计算研究所,哈马德·本·哈利法大学研究中心);MIT Computer Science and Artificial Intelligence Laboratory(麻省理工学院计算机科学与人工智能实验室)
尽管深度神经网络在自然语言处理(NLP)中取得了显著的进步,但其可解释性仍然是一个挑战。以前的工作主要集中在这些模型在表示级别上学习什么。我们进一步分解这种分析,并研究NLP任务中端到端神经模型学习的向量表示中的各个维度(神经元)。我们提出两种方法:语言相关性分析,基于监督方法提取与外部任务最相关的神经元,以及跨模型相关性分析,一种无监督方法提取与外部任务最相关的突出神经元。模型本身。我们通过消融识别出的神经元并重新评估两项任务的网络性能来评估我们的技术的有效性:神经机器翻译(NMT)和神经语言建模(NLM)。我们进一步提出了一个全面的神经元分析,旨在解决以下问题:i)如何本地化或分布的模型中的不同的语言属性?ii)某些神经元是否只具有某些特性而不具有其他特性?iii)信息在NMT和NLM中的分布是否更大或更小?以及iv)通过语言相关方法识别的神经元对整体任务有多重要?我们的代码作为NeuroX工具包的一部分公开提供(Dalvi et al. 2019)。