arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.13318cs.CLcs.LG
收藏

具有线性竞争单元的随机Transformer网络:应用于端到端SL转换

Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translation

Cyprus University of Technology(塞浦路斯理工大学);University of Patras(帕特雷大学);Rutgers University(罗格斯大学)

Andreas Voskou, Konstantinos P. Panousis, Dimitrios Kosmopoulos, Dimitris N. Metaxas, Sotirios Chatzis

摘要:

自动手语翻译(Automating sign language translation,缩写为ESTA)是一个具有挑战性的现实世界的应用。尽管其社会重要性,但该领域的研究进展仍然相当差。至关重要的是,现有的方法,产生可行的性能需要费力的获得光泽序列地面实况的可用性。在本文中,我们通过引入一个不需要明确使用注释的端到端模型来减弱这种需求;该模型只需要文本地面实况。这与现有的端到端模型形成鲜明对比,现有的端到端模型使用光泽序列地面实况,要么以在中间模型阶段识别的模态的形式,要么以并行输出过程的形式,与训练模型联合训练。我们的方法构成了一个具有新型层的Transformer网络,该网络结合了:(i)具有随机赢家采样的局部赢家通吃(LWTA)层,而不是传统的ReLU层,(ii)具有通过变分推理估计的后验分布的随机权重,以及(iii)在推理时的权重压缩技术,该技术利用估计的后验方差来执行大规模,几乎无损压缩。我们证明了我们的方法可以达到PHOENIX 2014 T基准测试中目前最好的BLEU-4分数,但不使用模型训练的注释,并且内存占用减少了70%以上。

英文摘要:

Automating sign language translation (SLT) is a challenging real world application. Despite its societal importance, though, research progress in the field remains rather poor. Crucially, existing methods that yield viable performance necessitate the availability of laborious to obtain gloss sequence groundtruth. In this paper, we attenuate this need, by introducing an end-to-end SLT model that does not entail explicit use of glosses; the model only needs text groundtruth. This is in stark contrast to existing end-to-end models that use gloss sequence groundtruth, either in the form of a modality that is recognized at an intermediate model stage, or in the form of a parallel output process, jointly trained with the SLT model. Our approach constitutes a Transformer network with a novel type of layers that combines: (i) local winner-takes-all (LWTA) layers with stochastic winner sampling, instead of conventional ReLU layers, (ii) stochastic weights with posterior distributions estimated via variational inference, and (iii) a weight compression technique at inference time that exploits estimated posterior variance to perform massive, almost lossless compression. We demonstrate that our approach can reach the currently best reported BLEU-4 score on the PHOENIX 2014T benchmark, but without making use of glosses for model training, and with a memory footprint reduced by more than 70%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑