可组合的视觉标记生成器：无生成器诊断的可学习性
Composable Visual Tokenizers with Generator-Free Diagnostics of Learnability
AI总结：
CompTok通过训练可组合的视觉标记生成器，提升图像生成的组合性和语义编辑能力，同时引入两个度量标准评估标记空间的可学习性。
摘要：
我们介绍了CompTok，一种用于学习视觉标记生成器的训练框架，其标记被增强以实现组合性。CompTok使用一种基于标记的扩散解码器。通过采用类似InfoGAN的目标函数，我们训练一个识别模型，利用解码图像来预测用于条件解码器的标记，从而强制解码器不忽略任何标记。为了促进组合控制，除了原始图像外，CompTok还训练在通过交换图像间标记子集生成的标记上，从而在解码器上实现更可控的标记组合。由于图像间的交换标记没有真实图像目标，我们通过对抗流正则化器应用流形约束，以保持无配对交换生成在自然图像分布上。所得到的标记生成器不仅在图像类别条件生成上实现了最先进的性能，还展示了通过在图像间交换标记以实现高级语义编辑的能力。此外，我们提出了两个度量标准，用于衡量标记空间的景观，这有助于描述标记的组合性以及生成器在该空间上学习该景观的难易程度。我们在实验中展示了CompTok能够改进这两个度量标准，并支持最先进的类别条件生成器。
英文摘要：
We introduce CompTok, a training framework for learning visual tokenizers whose tokens are enhanced for compositionality. CompTok uses a token-conditioned diffusion decoder. By employing an InfoGAN-style objective, where we train a recognition model to predict the tokens used to condition the diffusion decoder using the decoded images, we enforce the decoder to not ignore any of the tokens. To promote compositional control, besides the original images, CompTok also trains on tokens formed by swapping token subsets between images, enabling more compositional control of the token over the decoder. As the swapped tokens between images do not have ground truth image targets, we apply a manifold constraint via an adversarial flow regularizer to keep unpaired swap generations on the natural-image distribution. The resulting tokenizer not only achieves state-of-the-art performance on image class-conditioned generation, but also demonstrates properties such as swapping tokens between images to achieve high level semantic editing of an image. Additionally, we propose two metrics that measures the landscape of the token space that can be useful to describe not only the compositionality of the tokens, but also how easy to learn the landscape is for a generator to be trained on this space. We show in experiments that CompTok can improve on both of the metrics as well as supporting state-of-the-art generators for class conditioned generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
