X-Tokenizer: 一种用于视觉-语言-动作预训练的多模态动作分词器
X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining
AI总结：
提出X-Tokenizer，通过语义残差量化（SRQ）和掩码动作建模（MAM）将动作离散化为语义接口，在2.4M轨迹上预训练后提升VLA模型的多模态接地和长程任务性能。
摘要：
现代视觉-语言-动作（VLA）模型必须桥接预训练的视觉-语言推理和精确的连续机器人控制。现有的动作分词器主要为了重建而离散化动作，产生的编码保留了运动几何结构，但仅向主干网络提供弱语义监督。因此，我们将动作分词化不仅视为压缩，而是作为多模态推理与可执行控制之间的语义接口学习。为此，我们引入了X-Tokenizer，一种轻量级的编码器-语义残差量化（SRQ）-解码器架构，为多种机械臂形态提供共享的动作接口。其关键组件SRQ在残差向量量化上施加了非对称结构：第一层通过掩码动作建模（MAM）训练，形成捕获粗略运动意图的离散动作语言，而更深层则保持面向重建的残差，保留细粒度细节。为了进一步将动作标记与多模态语义对齐，X-Tokenizer通过与预训练基础模型的表示空间进行对比对齐以及下一帧视觉-语言特征预测进行预训练。在2.4M轨迹（2.0B动作帧）上预训练后，单个冻结的X-Tokenizer作为表示塑造的监督信号插入混合离散-连续VLA中。X-Tokenizer在真实世界聚合指标上达到最佳，并在RoboTwin 2.0模拟中表现强劲。在多模态接地（+13.5%）和长程任务（+8.25）上优于FAST，表明动作分词器作为VLA预训练的语义接口，而不仅仅是动作压缩。
英文摘要：
Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing codes that preserve motion geometry but provide only weak semantic supervision to the backbone. We therefore formulate action tokenization not as mere compression, but as semantic interface learning between multimodal reasoning and executable control. To this end, we introduce X-Tokenizer, a lightweight encoder-Semantic Residual Quantization (SRQ)-decoder architecture that provides a shared action interface across diverse robotic arm embodiments. Its key component, SRQ, imposes an asymmetric structure on residual vector quantization: the first level is trained with Masked Action Modeling (MAM) to form a discrete action language that captures coarse motion intent, while deeper levels remain reconstruction-oriented residuals that preserve fine-grained details. To further align action tokens with multimodal semantics, X-Tokenizer is pretrained with contrastive alignment to the representation space of a pretrained foundation model and with next-frame vision-language feature prediction. Pretrained on 2.4M trajectories (2.0B action frames), a single frozen X-Tokenizer plugs into a mixed discrete-continuous VLA as a representation-shaping supervision signal. X-Tokenizer achieves top real-world aggregate and strong RoboTwin 2.0 simulation results. Outperforming FAST in multimodal grounding (+13.5%) and long-horizon tasks (+8.25), it shows that action tokenizers serve as semantic interfaces for VLA pretraining beyond mere action compression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
