Transformers知道符号规则吗？如果他们知道，我们会知道吗？
Do Transformers know symbolic rules, and would we know if they did?
摘要：
为了提高NLP中使用的主要Transformer网络的可解释性，重要的是要将真正的符号规则与仅仅是关联的输入输出模式区分开来。然而，我们发现了几个不一致的“象征性”已在最近的NLP文献解释。为了缓解这个问题，我们提出了两个标准是最相关的，一个有关系统的内部架构和其他抽象规则和具体的输入身份之间的分离。从这个角度来看，我们严格审查以前的工作的象征性能力的Transformers，并认为结果是从根本上不确定的内在原因在实验设计。我们还认为，这个问题没有简单的解决办法，因为它在某种程度上出现在所有端到端的设置中。尽管如此，我们强调需要更有力的评估是否存在非象征性的解释，在看似象征性的任务的成功。为了促进这一点，我们在T5 Transformer上的四个序列建模任务上进行了两种实验设置：zero-shot泛化，以及在训练集和测试集之间翻转的类特定词汇的泛化。我们观察到，T5的概括是显着更强的序列到序列的任务比可比的分类任务。基于此，我们提出了一个迄今为止被忽视的分析，其中Transformer本身不需要是符号的，作为处理器的符号架构的一部分，作为外部存储器组件在输入和输出上操作。
英文摘要：
To improve the explainability of leading Transformer networks used in NLP, it is important to tease apart genuine symbolic rules from merely associative input-output patterns. However, we identify several inconsistencies in how ``symbolicity'' has been construed in recent NLP literature. To mitigate this problem, we propose two criteria to be the most relevant, one pertaining to a system's internal architecture and the other to the dissociation between abstract rules and specific input identities. From this perspective, we critically examine prior work on the symbolic capacities of Transformers, and deem the results to be fundamentally inconclusive for reasons inherent in experiment design. We further maintain that there is no simple fix to this problem, since it arises -- to an extent -- in all end-to-end settings. Nonetheless, we emphasize the need for more robust evaluation of whether non-symbolic explanations exist for success in seemingly symbolic tasks. To facilitate this, we experiment on four sequence modelling tasks on the T5 Transformer in two experiment settings: zero-shot generalization, and generalization across class-specific vocabularies flipped between the training and test set. We observe that T5's generalization is markedly stronger in sequence-to-sequence tasks than in comparable classification tasks. Based on this, we propose a thus far overlooked analysis, where the Transformer itself does not need to be symbolic to be part of a symbolic architecture as the processor, operating on the input and output as external memory components.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
