广义分类器真的适用于人类手绘草图吗？
Do Generalised Classifiers really work on Human Drawn Sketches?
AI总结：
本文首次将 CLIP 基础模型适配到草图理解，通过草图特定提示和抽象层次码本实现跨类别与跨抽象层次的广义草图分类，在零样本和少样本设置中超越现有方法。
摘要：
本文首次将大型基础模型与人类草图理解相结合。我们展示了这一结合所带来的成果——在广义草图表示学习（例如分类）方面的一种范式转变。这种泛化发生在两个层面：（i）跨未知类别的泛化（即开放集），以及（ii）跨越抽象层次的泛化（即好的和差的草图），这两者都是草图文献中尚未解决的紧迫挑战。我们的设计直观且以迁移 CLIP 已有的卓越泛化能力为核心，使草图受益于广义学习。我们首先通过一种新颖的栅格到矢量草图转换辅助头来学习草图特定提示，从而对原始 CLIP 模型进行“条件化”。重要的是，这使 CLIP 变得“草图感知”。随后，我们使 CLIP 对固有地不同的草图抽象层次变得敏锐。这是通过学习一个抽象特定提示偏置的码本来实现的，这些偏置的加权组合有助于跨抽象层次表示草图——低抽象边缘图、TU-Berlin 中的中等抽象草图以及 QuickDraw 中的高度抽象涂鸦。我们的框架在零样本和少样本设置中，以及在跨不同抽象边界的新颖设置中，均超越了流行的草图表示学习算法。
英文摘要：
This paper, for the first time, marries large foundation models with human sketch understanding. We demonstrate what this brings -- a paradigm shift in terms of generalised sketch representation learning (e.g., classification). This generalisation happens on two fronts: (i) generalisation across unknown categories (i.e., open-set), and (ii) generalisation traversing abstraction levels (i.e., good and bad sketches), both being timely challenges that remain unsolved in the sketch literature. Our design is intuitive and centred around transferring the already stellar generalisation ability of CLIP to benefit generalised learning for sketches. We first "condition" the vanilla CLIP model by learning sketch-specific prompts using a novel auxiliary head of raster to vector sketch conversion. This importantly makes CLIP "sketch-aware". We then make CLIP acute to the inherently different sketch abstraction levels. This is achieved by learning a codebook of abstraction-specific prompt biases, a weighted combination of which facilitates the representation of sketches across abstraction levels -- low abstract edge-maps, medium abstract sketches in TU-Berlin, and highly abstract doodles in QuickDraw. Our framework surpasses popular sketch representation learning algorithms in both zero-shot and few-shot setups and in novel settings across different abstraction boundaries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
