PartCraft：通过部件创作创意物体
PartCraft: Crafting Creative Objects by Parts
AI总结：
本文提出 PartCraft，支持用户按部件选择视觉概念，并通过无监督部件聚类、文本 token 编码、熵归一化注意力损失和瓶颈编码器实现细粒度且整体合理的创意生成。
摘要：
本文通过允许用户“选择”来推动生成式视觉 AI 中的创意控制。不同于传统的基于文本或草图的方法，我们首次允许用户在其创意创作中按部件选择视觉概念。其结果是细粒度生成，能够精确捕捉所选择的视觉概念，确保整体上忠实且合理的结果。为实现这一点，我们首先通过无监督特征聚类将物体解析为部件。随后，我们将部件编码为文本 token，并引入一种作用于这些 token 的、基于熵的归一化注意力损失。该损失设计使我们的模型能够学习关于物体部件组成的通用先验拓扑知识，并进一步泛化到新颖的部件组成，以确保生成结果在整体上看起来忠实。最后，我们采用瓶颈编码器来投影部件 token。这不仅提升了保真度，还通过利用共享知识并促进实例之间的信息交换来加快学习。论文和补充材料中的视觉结果展示了 PartCraft 在创作高度定制化、创新性作品方面的强大能力，例如“迷人的”创意鸟类。代码已发布于 https://github.com/kamwoh/partcraft。
英文摘要：
This paper propels creative control in generative visual AI by allowing users to "select". Departing from traditional text or sketch-based methods, we for the first time allow users to choose visual concepts by parts for their creative endeavors. The outcome is fine-grained generation that precisely captures selected visual concepts, ensuring a holistically faithful and plausible result. To achieve this, we first parse objects into parts through unsupervised feature clustering. Then, we encode parts into text tokens and introduce an entropy-based normalized attention loss that operates on them. This loss design enables our model to learn generic prior topology knowledge about object's part composition, and further generalize to novel part compositions to ensure the generation looks holistically faithful. Lastly, we employ a bottleneck encoder to project the part tokens. This not only enhances fidelity but also accelerates learning, by leveraging shared knowledge and facilitating information exchange among instances. Visual results in the paper and supplementary material showcase the compelling power of PartCraft in crafting highly customized, innovative creations, exemplified by the "charming" and creative birds. Code is released at https://github.com/kamwoh/partcraft.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
