arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.09843cs.CV
收藏

Kelix技术报告

Kelix Technical Report

Kuaishou Technology(快手科技)
Boyang Ding, Chenglong Chu, Dunju Zang, Han Li, Jiangxia Cao, Kun Gai, Muhao Wei, Ruiming Tang, Shiyao Wang, Siyang Mao, Xinchen Luo, Yahui Liu, Zhixin Ling, Zh… 展开作者

Boyang Ding, Chenglong Chu, Dunju Zang, Han Li, Jiangxia Cao, Kun Gai, Muhao Wei, Ruiming Tang, Shiyao Wang, Siyang Mao, Xinchen Luo, Yahui Liu, Zhixin Ling, Zhuoran Yang, Ziming Li, Chengru Song, Guorui Zhou, Guowang Zhang, Hao Peng, Hao Wang, Jiaxin Deng, Jin Ouyang, Jinghao Zhang, Lejian Ren, Qianqian Wang, Qigen Hu, Tao Wang, Xingmei Wang, Yiping Yang, Zixing Zhang, Ziqi Wang

AI总结:

Kelix是一种完全离散的自回归统一模型,旨在缩小离散和连续视觉表示之间的理解差距。

摘要:

自回归大语言模型(LLMs)通过将多样任务表示为离散自然语言标记序列,并通过下一个标记预测进行训练,能够良好扩展。这种范式通过自监督统一了理解和生成。将此范式扩展到多模态数据需要跨模态的共享离散表示。然而,大多数视觉-语言模型(VLMs)仍然依赖于混合接口:离散文本标记与连续视觉Transformer(ViT)特征的结合。由于监督主要由文本驱动,这些模型往往偏向于理解,无法充分利用大规模自监督学习在非文本数据上的潜力。最近的研究探索了离散视觉标记化,以实现完全自回归的多模态建模,显示出向统一理解和生成迈进的积极进展。然而,现有的离散视觉标记由于代码容量有限,常常丢失信息,导致理解能力明显弱于连续特征VLMs。我们提出了Kelix,一个完全离散的自回归统一模型,缩小了离散和连续视觉表示之间的理解差距。

英文摘要:

Autoregressive large language models (LLMs) scale well by expressing diverse tasks as sequences of discrete natural-language tokens and training with next-token prediction, which unifies comprehension and generation under self-supervision. Extending this paradigm to multimodal data requires a shared, discrete representation across modalities. However, most vision-language models (VLMs) still rely on a hybrid interface: discrete text tokens paired with continuous Vision Transformer (ViT) features. Because supervision is largely text-driven, these models are often biased toward understanding and cannot fully leverage large-scale self-supervised learning on non-text data. Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling, showing promising progress toward unified understanding and generation. Yet existing discrete vision tokens frequently lose information due to limited code capacity, resulting in noticeably weaker understanding than continuous-feature VLMs. We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑