arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2003.02314cs.CVcs.LGeess.IV
收藏

孔洞几何形状对In-Painting网络相对鲁棒性影响的实证研究

The Impact of Hole Geometry on Relative Robustness of In-Painting Networks: An Empirical Study

Futurewei, Inc.(Futurewei公司);Santa Clara(圣克拉拉(这里作为城市名,非机构,所以不提取))

Masood S. Mortazavi, Ning Yan

摘要:

图像修复网络使用现有的像素来生成适当的像素,以填充放置在图像部分上的“洞”。2-D in-painting网络的输入通常包括(1)三通道2-D图像,以及(2)用于在该图像中进行补绘的“孔”的附加通道。在本文中,我们研究了一个给定的在绘画神经网络对孔几何分布的变化的鲁棒性。我们观察到,修复网络的鲁棒性取决于在其训练期间呈现给它的孔几何形状的概率分布函数(PDF),即使(在训练和测试中)使用的底层图像数据集没有改变。我们开发了一种实验方法来测试和评估相对鲁棒性的修补网络对四种不同的孔几何PDF。我们研究了一些假设,涉及(1)修补网络对用于其训练的孔分布的自然偏差,(2)底层数据集区分相对鲁棒性的能力,因为孔分布在训练测试(交叉比较)网格中变化,以及(3)孔和图像数据集中边缘的方向分布的影响。我们目前的L1,PSNR和SSIM质量指标的结果,并制定了一个具体的措施,相对在绘画的鲁棒性,用于交叉比较网格的基础上,这些质量指标。(One可以在该相对度量中结合其他质量度量。本文的实验工作是对“填补空白”神经网络的灵敏度、鲁棒性和正则化对孔洞“几何”PDF的更广泛和更深入研究的第一步,并建议在这一领域进行进一步研究。

英文摘要:

In-painting networks use existing pixels to generate appropriate pixels to fill "holes" placed on parts of an image. A 2-D in-painting network's input usually consists of (1) a three-channel 2-D image, and (2) an additional channel for the "holes" to be in-painted in that image. In this paper, we study the robustness of a given in-painting neural network against variations in hole geometry distributions. We observe that the robustness of an in-painting network is dependent on the probability distribution function (PDF) of the hole geometry presented to it during its training even if the underlying image dataset used (in training and testing) does not alter. We develop an experimental methodology for testing and evaluating relative robustness of in-painting networks against four different kinds of hole geometry PDFs. We examine a number of hypothesis regarding (1) the natural bias of in-painting networks to the hole distribution used for their training, (2) the underlying dataset's ability to differentiate relative robustness as hole distributions vary in a train-test (cross-comparison) grid, and (3) the impact of the directional distribution of edges in the holes and in the image dataset. We present results for L1, PSNR and SSIM quality metrics and develop a specific measure of relative in-painting robustness to be used in cross-comparison grids based on these quality metrics. (One can incorporate other quality metrics in this relative measure.) The empirical work reported here is an initial step in a broader and deeper investigation of "filling the blank" neural networks' sensitivity, robustness and regularization with respect to hole "geometry" PDFs, and it suggests further research in this domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑