arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02597cs.CLcs.AIcs.DCcs.LG
收藏

GPUTOK: 基于GPU的字节级BPE分词

GPUTOK: GPU Accelerated Byte Level BPE Tokenization

New York University(纽约大学)

Venu Gopal Kadamba, Kanishkha Jaisankar

AI总结:

GPUTOK提出基于GPU的字节级BPE分词器,通过优化算法和内存管理,显著提升分词速度,同时保持输出质量,适用于长上下文推理任务。

摘要:

随着大语言模型朝着百万token上下文窗口发展,CPU分词器成为主要的瓶颈,因为它们逐步处理文本,而强大的GPU则被闲置。我们构建了一个基于GPU的字节级BPE分词器,遵循GPT-2的合并规则。它包括一个基本的BlockBPE风格内核和一个更快、优化的版本,该版本使用cuCollections静态映射、CUB归约和pybind11接口进行Python编程。在WikiText103序列上,最多131k个token,优化的GPU分词器生成的token与CPU版本相同,对于最长的输入,其速度比tiktoken快约1.7倍,比HuggingFace GPT-2分词器快约7.6倍。Nsight分析显示,70-80%的CUDA API时间用于内存分配,因此添加内存池应带来最大的速度提升。在使用WikiText103提示的生成任务测试中,我们的GPU分词器的输出在相似性和重叠度指标上与tiktoken和HuggingFace GPT-2相差约1个百分点,这意味着它在保持输出质量的同时,使长上下文推理更加实用。

英文摘要:

As large language models move toward million-token context windows, CPU tokenizers become a major slowdown because they process text one step at a time while powerful GPUs sit unused. We built a GPU-based byte-level BPE tokenizer that follows GPT-2's merge rules. It includes a basic BlockBPE-style kernel and a faster, optimized version that uses cuCollections static map, CUB reductions, and a pybind11 interface for Python. On WikiText103 sequences up to 131k tokens, the optimized GPU tokenizer produces the same tokens as a CPU version and, for the longest inputs, is about 1.7x faster than tiktoken and about 7.6x faster than the HuggingFace GPT-2 tokenizer. Nsight profiling shows that 70-80% of CUDA API time goes to memory allocation, so adding memory pooling should give the biggest speed boost next. Tests on generation tasks using WikiText103 prompts show that our GPU tokenizer's outputs stay within about one percentage point of tiktoken and HuggingFace GPT-2 on similarity and overlap metrics, meaning it keeps output quality while making long-context inference more practical.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑