arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.03116cs.CVcs.GRcs.IRcs.LG
收藏

基于内容的深度生成模型搜索

Content-Based Search for Deep Generative Models

Carnegie Mellon University(卡内基梅隆大学);Northeastern University(东北大学)

Daohan Lu, Sheng-Yu Wang, Nupur Kumari, Rohan Agarwal, Mia Tang, David Bau, Jun-Yan Zhu

AI总结:

本文提出基于内容的深度生成模型搜索任务,通过优化模型生成相似内容的概率,并利用对比学习框架适应图像、草图和文本等多模态查询,在新建的 Generative Model Zoo 基准上取得了优于基线的检索效果。

摘要:

定制化和预训练生成模型的日益增多使得用户无法全面了解现有的每一个模型。为了满足这一需求,我们引入了基于内容的模型搜索任务:给定一个查询和大量生成模型,寻找最匹配查询的模型。由于每个生成模型都会产生一个图像分布,我们将搜索任务表述为一个优化问题,以选择最有可能生成与查询相似内容的模型。我们引入了一种公式,用于在给定来自不同模态(例如图像、草图和文本)的查询时近似此概率。此外,我们提出了一个用于模型检索的对比学习框架,该框架学习为各种查询模态调整特征。我们证明,我们的方法在 Generative Model Zoo(我们为模型检索任务创建的新基准)上优于几个基线。

英文摘要:

The growing proliferation of customized and pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, finding the models that best match the query. As each generative model produces a distribution of images, we formulate the search task as an optimization problem to select the model with the highest probability of generating similar content as the query. We introduce a formulation to approximate this probability given the query from different modalities, e.g., image, sketch, and text. Furthermore, we propose a contrastive learning framework for model retrieval, which learns to adapt features for various query modalities. We demonstrate that our method outperforms several baselines on Generative Model Zoo, a new benchmark we create for the model retrieval task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑