图像与对象级特征协同着色
Collaboration among Image and Object Level Features for Image Colourisation
摘要:
图像着色是一个病态问题,有多个正确的解决方案,这取决于输入数据中存在的上下文和对象实例。以前的方法要么通过需要强烈的用户交互,要么通过利用卷积神经网络(CNN)学习图像级(上下文)特征的能力来解决这个问题。然而,获得人类提示并不总是可行的,并且CNN本身无法学习对象级语义,除非考虑使用监督预训练的多个模型。在这项工作中,我们提出了一个名为UCapsNet的网络,它将通过卷积获得的图像级特征和通过胶囊捕获的对象级特征分开。然后,通过跳过不同层上的连接,我们加强了这些解开因素之间的合作,以产生高质量和合理的图像着色。我们提出的问题作为一个分类任务,可以通过一个完全自我监督的方法来解决,因此不需要人为的努力。三个基准数据集上的实验结果表明,我们的方法优于现有的方法在标准的质量指标,并实现了最先进的性能图像着色。一个大规模的用户研究表明,我们的方法是优于现有的解决方案。
英文摘要:
Image colourisation is an ill-posed problem, with multiple correct solutions which depend on the context and object instances present in the input datum. Previous approaches attacked the problem either by requiring intense user interactions or by exploiting the ability of convolutional neural networks (CNNs) in learning image level (context) features. However, obtaining human hints is not always feasible and CNNs alone are not able to learn object-level semantics unless multiple models pretrained with supervision are considered. In this work, we propose a single network, named UCapsNet, that separate image-level features obtained through convolutions and object-level features captured by means of capsules. Then, by skip connections over different layers, we enforce collaboration between such disentangling factors to produce high quality and plausible image colourisation. We pose the problem as a classification task that can be addressed by a fully self-supervised approach, thus requires no human effort. Experimental results on three benchmark datasets show that our approach outperforms existing methods on standard quality metrics and achieves a state of the art performances on image colourisation. A large scale user study shows that our method is preferred over existing solutions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。