arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1908.01665cs.CL
收藏

预测行为以帮助预测翻译

Predicting Actions to Help Predict Translations

Imperial College London(帝国理工学院);Sheffield University(谢菲尔德大学)

Zixiu Wu, Julia Ive, Josiah Wang, Pranava Madhyastha, Lucia Specia

摘要:

我们使用最先进的基于transformer的多模态方法来解决How2数据集上的文本翻译任务。我们问自己的问题是视觉特征是否可以支持翻译过程,特别是,鉴于这是一个从视频中提取的数据集,我们专注于动作的翻译,我们认为这在当前用于多模态翻译的静态图像-文本数据集中捕获得很差。为此,我们从视频中提取了不同类型的动作特征,并通过测试这种视觉信息是否可以提高翻译质量来仔细研究这种视觉信息的帮助程度,当与(i)原始文本和(ii)原始文本结合使用时,与动作相关的单词(或所有动词)被掩盖了。后者是一种模拟,可以帮助我们在文本没有提供足够的动作上下文或输入文本中存在噪声的情况下评估图像的效用。

英文摘要:

We address the task of text translation on the How2 dataset using a state of the art transformer-based multimodal approach. The question we ask ourselves is whether visual features can support the translation process, in particular, given that this is a dataset extracted from videos, we focus on the translation of actions, which we believe are poorly captured in current static image-text datasets currently used for multimodal translation. For that purpose, we extract different types of action features from the videos and carefully investigate how helpful this visual information is by testing whether it can increase translation quality when used in conjunction with (i) the original text and (ii) the original text where action-related words (or all verbs) are masked out. The latter is a simulation that helps us assess the utility of the image in cases where the text does not provide enough context about the action, or in the presence of noise in the input text.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑