arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.02548cs.CV
收藏

GeNIe:通过扩散生成困难负样本图像

GeNIe: Generative Hard Negative Images Through Diffusion

University of California, Davis(加州大学戴维斯分校);Delft University of Technology(代尔夫特理工大学);Shell Global Solutions International B.V.(壳牌全球解决方案国际有限公司)

Soroush Abbasi Koohpayegani, Anuj Singh, K L Navaneet, Hamed Pirsiavash, Hadi Jamali-Rad

AI总结:

GeNIe利用潜在扩散模型结合源图像与目标文本生成困难负样本,并通过自适应噪声调整(GeNIe-Ada)提升少样本和长尾分布下的分类性能。

摘要:

数据增强在训练深度模型中至关重要,可防止模型对有限数据过拟合。生成式AI(如扩散模型)的最新进展,使得能够产生类似自然图像数据的更复杂增强技术成为可能。我们提出GeNIe,一种新颖的增强方法,利用以文本提示为条件的潜在扩散模型,将两个对比数据点(来自源类别的图像和来自目标类别的文本提示)结合起来,生成具有挑战性的增强样本。为实现此目标,我们调整噪声水平(等效于扩散迭代次数),以确保生成的图像保留源图像的低层和背景特征,同时代表目标类别,从而为源类别产生困难负样本。我们进一步通过自适应地调整每个图像的噪声水平选择(称为GeNIe-Ada)来自动化和增强GeNIe,从而带来进一步的性能提升。我们在少样本和长尾分布设置下的大量实验,证明了我们新颖增强方法的有效性及其相对于先前技术的优越性能。我们的代码可在 https://github.com/UCDvision/GeNIe 获取。

英文摘要:

Data augmentation is crucial in training deep models, preventing them from overfitting to limited data. Recent advances in generative AI, e.g., diffusion models, have enabled more sophisticated augmentation techniques that produce data resembling natural images. We introduce GeNIe a novel augmentation method which leverages a latent diffusion model conditioned on a text prompt to combine two contrasting data points (an image from the source category and a text prompt from the target category) to generate challenging augmentations. To achieve this, we adjust the noise level (equivalently, number of diffusion iterations) to ensure the generated image retains low-level and background features from the source image while representing the target category, resulting in a hard negative sample for the source category. We further automate and enhance GeNIe by adaptively adjusting the noise level selection on a per image basis (coined as GeNIe-Ada), leading to further performance improvements. Our extensive experiments, in both few-shot and long-tail distribution settings, demonstrate the effectiveness of our novel augmentation method and its superior performance over the prior art. Our code is available at: https://github.com/UCDvision/GeNIe

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑