多通道生成语言模型:学习通道内和通道间所有可能的因子分解
Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels
摘要:
通道对应于潜在意义的视点或变换。英语和法语中的一对平行句子表达了相同的潜在含义,但通过两个独立的渠道对应于他们的语言。本文提出了多通道生成语言模型(MGLM)。MGLM是一种跨渠道的生成式联合分销模型。MGLM在所有通道内和跨所有通道对所有可能的因子分解进行边缘化。MGLM赋予灵活的推理,包括无条件生成,有条件生成(其中观察到1个通道,并生成其他通道)和部分观察生成(其中不完整的观察分布在所有通道上)。我们使用包含英语、法语、捷克语和德语的Multi30K数据集进行实验。我们展示了无条件,有条件和部分条件生成的实验。我们提供从生成联合分布无条件采样的定性样本。我们还定量分析了质量多样性的权衡,发现MGLM优于传统的双语歧视模型。
英文摘要:
A channel corresponds to a viewpoint or transformation of an underlying meaning. A pair of parallel sentences in English and French express the same underlying meaning, but through two separate channels corresponding to their languages. In this work, we present the Multichannel Generative Language Model (MGLM). MGLM is a generative joint distribution model over channels. MGLM marginalizes over all possible factorizations within and across all channels. MGLM endows flexible inference, including unconditional generation, conditional generation (where 1 channel is observed and other channels are generated), and partially observed generation (where incomplete observations are spread across all the channels). We experiment with the Multi30K dataset containing English, French, Czech, and German. We demonstrate experiments with unconditional, conditional, and partially conditional generation. We provide qualitative samples sampled unconditionally from the generative joint distribution. We also quantitatively analyze the quality-diversity trade-offs and find MGLM outperforms traditional bilingual discriminative models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。