arXiv 2306.007512023-06-01cs.CLcs.LG

可微树操作促进组合泛化

Differentiable Tree Operations Promote Compositional Generalization

Johns Hopkins University（约翰斯·霍普金斯大学）；Mila（米拉研究所）；Université de Montreal（蒙特利尔大学）；University of Edinburgh（爱丁堡大学）；Microsoft Research（微软研究院）

Paul Soulos, Edward Hu, Kate McCurdy, Yunmo Chen, Roland Fernandez, Paul Smolensky, Jianfeng Gao

备注：: ICML 2023. Code available at https://github.com/psoulos/dtm

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出可微树解释器与DTM架构，将离散树操作转为可微张量计算并端到端学习操作序列，在合成语义解析和语言生成的分布外组合泛化上达到100%，显著优于现有基线且保持可解释性。

摘要：

在结构到结构转换任务的背景下，学习离散符号操作序列因其不可微性而面临重大挑战。为促进这些符号序列的学习，我们引入了一种可微树解释器，它将高层符号树操作编译为张量上的亚符号矩阵操作。我们提出了一种新颖的可微树机（Differentiable Tree Machine，DTM）架构，该架构将我们的解释器与外部存储器以及一个智能体相集成，该智能体学习按顺序选择树操作，以端到端方式执行目标转换。关于合成语义解析和语言生成任务上的分布外组合泛化，DTM达到了100%，而Transformer、Tree Transformer、LSTM和Tree2Tree LSTM等现有基线的成绩低于30%。除完美性能外，DTM还保持了高度的可解释性。

英文摘要：

In the context of structure-to-structure transformation tasks, learning sequences of discrete symbolic operations poses significant challenges due to their non-differentiability. To facilitate the learning of these symbolic sequences, we introduce a differentiable tree interpreter that compiles high-level symbolic tree operations into subsymbolic matrix operations on tensors. We present a novel Differentiable Tree Machine (DTM) architecture that integrates our interpreter with an external memory and an agent that learns to sequentially select tree operations to execute the target transformation in an end-to-end manner. With respect to out-of-distribution compositional generalization on synthetic semantic parsing and language generation tasks, DTM achieves 100% while existing baselines such as Transformer, Tree Transformer, LSTM, and Tree2Tree LSTM achieve less than 30%. DTM remains highly interpretable in addition to its perfect performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。