arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.10036cs.CV
收藏

Visual DNA:使用神经元激活分布表示和比较图像

Visual DNA: Representing and Comparing Images using Distributions of Neuron Activations

University of Oxford(牛津大学)

Benjamin Ramtoula, Matthew Gadd, Paul Newman, Daniele De Martini

AI总结:

本文提出Visual DNA,用冻结特征提取器的神经元激活分布紧凑表示图像与数据集,支持可定制的细粒度差异比较,并在多类视觉任务和数据集上验证其有效性。

摘要:

选择合适的数据集在现代计算机视觉中至关重要。然而,目前并不存在通用工具来评估两个数据集之间的差异程度。为此,我们提出使用神经元激活分布(Distributions of Neuron Activations,DNA)来表示图像,并进一步表示数据集。DNA会为预训练特征提取器中神经元的激活拟合各种分布,例如直方图或高斯分布;我们将待表示的图像输入该特征提取器。该提取器对所有数据集均保持冻结,我们依赖其在特征空间中普遍具备的表达能力。通过比较两个DNA,我们可以评估两个数据集之间的差异程度,并对感兴趣的比较属性进行细粒度控制,从而能够定制距离度量方式,以适应当前任务的要求。此外,DNA非常紧凑,能够用不到15兆字节表示任意规模的数据集。我们通过评估DNA在多项任务中的适用性来证明其价值,这些任务包括条件数据集比较、合成图像评估和迁移学习,并涵盖从合成猫图像到名人面孔和城市驾驶场景等多样化数据集。

英文摘要:

Selecting appropriate datasets is critical in modern computer vision. However, no general-purpose tools exist to evaluate the extent to which two datasets differ. For this, we propose representing images - and by extension datasets - using Distributions of Neuron Activations (DNAs). DNAs fit distributions, such as histograms or Gaussians, to activations of neurons in a pre-trained feature extractor through which we pass the image(s) to represent. This extractor is frozen for all datasets, and we rely on its generally expressive power in feature space. By comparing two DNAs, we can evaluate the extent to which two datasets differ with granular control over the comparison attributes of interest, providing the ability to customise the way distances are measured to suit the requirements of the task at hand. Furthermore, DNAs are compact, representing datasets of any size with less than 15 megabytes. We demonstrate the value of DNAs by evaluating their applicability on several tasks, including conditional dataset comparison, synthetic image evaluation, and transfer learning, and across diverse datasets, ranging from synthetic cat images to celebrity faces and urban driving scenes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑