用于重构漫画的密集多任务学习
Dense Multitask Learning to Reconfigure Comics
AI总结：
针对漫画缺乏密集标注且风格布局差异大的问题，提出基于 vision transformer 与域可迁移注意力的 MTL 模型，借助无监督图像转换利用真实标注，并结合重定向方法实现漫画重构。
摘要：
在本文中，我们开发了一个多任务学习（MultiTask Learning，MTL）模型，用于对漫画分镜进行密集预测，进而通过协助作者完成重构叙事的任务，促进漫画从一种出版渠道向另一种出版渠道的迁移。我们的 MTL 方法能够成功识别漫画分镜中的语义单元以及其中蕴含的 3D 概念。这是一个极具挑战性的问题，因为漫画包含因作者创作过程而异的不同艺术风格、插画、布局和物体尺度。通常，基于图像的密集预测技术需要大量语料数据。因此，由于缺乏漫画图像的真值密集标注，在漫画领域寻找密集预测的自动化解决方案变得更加困难。为应对这些挑战，我们提出了以下解决方案：1）我们利用一种常用的无监督图像到图像转换策略，使我们能够使用大量真实世界标注；2）我们利用转换结果开发多任务方法，该方法基于 vision transformer 主干和一个域可迁移注意力模块；3）我们研究了将我们的 MTL 密集预测方法与现有重定向方法相结合以重构漫画的可行性。
英文摘要：
In this paper, we develop a MultiTask Learning (MTL) model to achieve dense predictions for comics panels to, in turn, facilitate the transfer of comics from one publication channel to another by assisting authors in the task of reconfiguring their narratives. Our MTL method can successfully identify the semantic units as well as the embedded notion of 3D in comic panels. This is a significantly challenging problem because comics comprise disparate artistic styles, illustrations, layouts, and object scales that depend on the authors creative process. Typically, dense image-based prediction techniques require a large corpus of data. Finding an automated solution for dense prediction in the comics domain, therefore, becomes more difficult with the lack of ground-truth dense annotations for the comics images. To address these challenges, we develop the following solutions: 1) we leverage a commonly-used strategy known as unsupervised image-to-image translation, which allows us to utilize a large corpus of real-world annotations; 2) we utilize the results of the translations to develop our multitasking approach that is based on a vision transformer backbone and a domain transferable attention module; 3) we study the feasibility of integrating our MTL dense-prediction method with an existing retargeting method, thereby reconfiguring comics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
