arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.33540cs.CVcs.LG
收藏

基于Patch衍生标记与注册标记的比较:可变长度图像标记器

Patch-Derived or Register Tokens? Comparing Variable-Length Image Tokenizers

FZI Research Center for Information Technology(FZI信息技术研究中心);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)

Philipp Schenkel, Tobias Schürmann, Omer Sahin Tas

AI总结:

该研究对比Patch衍生与注册两种可变长度图像标记器,发现极端压缩时Patch衍生标记重建更优、下游生成任务中注册标记更优,加入与预训练视觉编码器的表示对齐可提升Patch衍生标记性能。

摘要:

图像标记器将像素转换为紧凑的潜在序列,以实现高效的生成式建模。大多数方法将潜在特征排列在降采样的二维网格上,使序列长度与图像分辨率相关联。我们比较两种可变长度策略:计算高效的Patch衍生标记,其继承显式空间组织;以及计算要求更高的注册标记,无预定义空间布局。重建结果显示,在极端压缩下,Patch衍生标记的性能显著更好;在中等预算下,注册标记的重建质量略优;在高标记预算下,两种方法的重建质量相近。对于下游生成式任务,注册标记的性能优于Patch衍生标记。添加与外部预训练视觉编码器的表示对齐,可大幅提升Patch衍生标记的生成性能,使其在不同标记预算下能匹配或超越注册标记。我们进一步发现,两种方法均学习到空间局部表示:Patch衍生标记基本保留其显式施加的空间组织,而注册标记虽无预定义布局,却形成了隐式的二维空间结构。表示对齐会削弱这种空间组织约束,使单个标记能发挥更强的非局部影响。

英文摘要:

Image tokenizers convert pixels into compact latent sequences for efficient generative modeling. Most approaches arrange latents on a downsampled two-dimensional grid, tying sequence length to image resolution. We compare two variable-length strategies: computationally efficient patch-derived tokens, which inherit explicit spatial organization, and more demanding register tokens, which have no predefined spatial layout. Our reconstruction results reveal that patch-derived tokens perform substantially better under extreme compression, whereas register tokens achieve slightly better reconstruction quality at intermediate budgets. At high token budgets, both approaches achieve similar reconstruction quality. For downstream generative tasks, register tokens outperform patch-derived tokens. Adding representation alignment with external pretrained visual encoders substantially improves the generative performance of patch-derived tokens, allowing them to match or outperform register tokens depending on the token budget. We further find that both approaches learn spatially localized representations. Patch-derived tokens largely retain their explicitly imposed spatial organization, while register tokens develop an implicit two-dimensional spatial structure despite having no predefined layout. Representation alignment reduces this spatial organizational constraint, enabling individual tokens to exert a stronger non-local influence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑