作为词汇对称性的组合性
Compositionality as Lexical Symmetry
AI总结：
本文提出LEXSYM，将组合性定义为数据分布对称性约束，通过自动发现数据变换并增强训练数据，使普通神经序列模型在语义解析、指令跟随和视觉问答等任务上达到或超越专用模型。
摘要：
在语义解析、指令跟随和问答等任务中，标准深度网络难以从小数据集实现组合泛化。许多现有方法通过强制句子解释的组合过程的模型架构来克服这一局限。本文提出一种领域通用且与模型无关的组合性表述，将组合性视为数据分布对称性上的约束，而非模型上的约束。非正式地说，我们证明：只要一个任务能够被组合模型解决，就存在一种相应的数据增强方案——一种将示例转换为其他良构示例的过程——能够对任何为解决同一任务而训练的模型赋予组合归纳偏置。我们描述了一种名为LEXSYM的过程，该过程自动发现这些变换，然后将其应用于普通神经序列模型的训练数据。与现有的组合数据增强过程不同，LEXSYM可以无关地部署在文本、结构化数据甚至图像上。在COGS语义解析、SCAN和ALCHEMY指令跟随以及CLEVR-COGENT视觉问答数据集上，它达到或超越了最先进的、任务特定的模型。
英文摘要：
In tasks like semantic parsing, instruction following, and question answering, standard deep networks fail to generalize compositionally from small datasets. Many existing approaches overcome this limitation with model architectures that enforce a compositional process of sentence interpretation. In this paper, we present a domain-general and model-agnostic formulation of compositionality as a constraint on symmetries of data distributions rather than models. Informally, we prove that whenever a task can be solved by a compositional model, there is a corresponding data augmentation scheme -- a procedure for transforming examples into other well formed examples -- that imparts compositional inductive bias on any model trained to solve the same task. We describe a procedure called LEXSYM that discovers these transformations automatically, then applies them to training data for ordinary neural sequence models. Unlike existing compositional data augmentation procedures, LEXSYM can be deployed agnostically across text, structured data, and even images. It matches or surpasses state-of-the-art, task-specific models on COGS semantic parsing, SCAN and ALCHEMY instruction following, and CLEVR-COGENT visual question answering datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
