Speech tokenizers are a key building block of fully discrete Speech LLMs. Existing tokenizers either prioritize semantic encoding, fuse semantic content with acoustic style inseparably, or achieve incomplete semantic-acoustic disentanglement. To achieve better disentanglement, we propose DSA-Tokenizer, which explicitly disentangles speech into discrete semantic and acoustic tokens via distinct optimization constraints. Specifically, semantic tokens are supervised by ASR to capture linguistic content, while acoustic tokens focus on mel-spectrograms restoration to encode style. We further introduce a hierarchical Flow Matching decoder and a joint reconstruction-context inpainting training strategy, allowing the model to support both high-fidelity reconstruction and cross-utterance voice clone. To speed up inference, we distill the dit decoder to 4-step inference and improve synthesis quality with GAN fine-tuning. Experiments demonstrate that DSA-Tokenizer provides strong semantic-acoustic disentanglement, reliable controllable voice cloning, and efficient high-fidelity generation with low WER/CER. Moreover, our results suggest that disentangled tokenization provides a more effective interface for downstream large-model speech generation. Audio samples are avaialble at https://anonymous.4open.science/w/DSA_Tokenizer_demo/
Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs
Saudi Data & AI Authority (SDAIA)(沙特数据与人工智能局);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学);Princess Nourah Bint Abdulrahman University (PNU)(普罗瑟·努拉·本·阿卜杜勒拉赫曼大学)
Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models
Princess Nourah Bint Abdulrahman University(普里丝琴·努拉·本·阿卜杜勒拉赫曼大学);Saudi Data & AI Authority (SDAIA)(沙特数据与人工智能管理局);University of Alberta(阿尔伯塔大学);Dialpad(Dialpad公司);York University(约克大学);Qatar Computing Research Institute(卡塔尔计算研究所);Amazon AGI(亚马逊人工智能研究所)
Beyond Fertility: Analyzing STRR as a Metric for Multilingual Tokenization Evaluation
University of Alberta(阿尔伯塔大学);Princess Nourah Bint Abdulrahman University(努拉·阿卜杜勒拉赫曼王后大学);Dialpad Qatar Computing Research Institute(阿联酋迪拜计算研究所);Amazon AGI(亚马逊人工智能研究所)
MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies
Qatar Computing Research Institute (QCRI)(卡塔尔计算研究所);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Technical University of Berlin(柏林工业大学)
MelTok: 2D Tokenization for Single-Codebook Audio Compression
International Digital Economy Academy (IDEA)(国际数字经济学院);Chinese University of Hong Kong(香港中文大学);Shenzhen Graduate School, Peking University(北京大学深圳研究生院);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生学院)