arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.14271cs.CL
收藏

走向全自动漫画翻译

Towards Fully Automated Manga Translation

Mantra Inc.(Mantra公司);The University of Tokyo(东京大学)

Ryota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, Yusuke Matsui

摘要:

我们解决了日本漫画的机器翻译问题。漫画翻译涉及到机器翻译中的两个重要问题:上下文感知和多模态翻译。由于漫画中的文本和图像以非结构化的方式混合在一起,因此从图像中获取上下文对于漫画翻译至关重要。然而,如何从图像中提取上下文并将其集成到机器翻译模型中仍然是一个悬而未决的问题。此外,目前还没有用于训练和评估这种模型的语料库和基准。本文从以下四个方面为漫画翻译研究奠定了基础。首先,我们提出了多模态上下文感知翻译框架。我们是第一个将从漫画图像获得的上下文信息。它使我们能够翻译对话气泡中的文本,如果不使用上下文信息(例如,其他对话气泡中的文本、说话者的性别等)。其次,为了训练模型,我们提出了一种自动语料库构建的方法,从原始漫画和他们的翻译对,通过这种方法可以构建大型平行语料库,而无需任何手动标注。第三,我们创建了一个新的标准来评估漫画翻译。最后,在我们提出的方法之上,我们设计了第一个全面的全自动漫画翻译系统。

英文摘要:

We tackle the problem of machine translation of manga, Japanese comics. Manga translation involves two important problems in machine translation: context-aware and multimodal translation. Since text and images are mixed up in an unstructured fashion in Manga, obtaining context from the image is essential for manga translation. However, it is still an open problem how to extract context from image and integrate into MT models. In addition, corpus and benchmarks to train and evaluate such model is currently unavailable. In this paper, we make the following four contributions that establishes the foundation of manga translation research. First, we propose multimodal context-aware translation framework. We are the first to incorporate context information obtained from manga image. It enables us to translate texts in speech bubbles that cannot be translated without using context information (e.g., texts in other speech bubbles, gender of speakers, etc.). Second, for training the model, we propose the approach to automatic corpus construction from pairs of original manga and their translations, by which large parallel corpus can be constructed without any manual labeling. Third, we created a new benchmark to evaluate manga translation. Finally, on top of our proposed methods, we devised a first comprehensive system for fully automated manga translation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑