arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02875cs.CL
收藏

重新审视语言模型在数据压缩中的应用

Revisiting Data Compression with Language Modeling

Chen-Han Tsai

AI总结:

本文探讨了使用大型语言模型进行数据压缩的潜力,展示了在enwik9数据集上达到18%的调整压缩率,并探讨了LLM在压缩非英语数据、代码和字节流序列中的应用。

摘要:

在本报告中,我们研究了大型语言模型(LLM)在数据压缩任务中的潜在应用。先前的研究已展示了将LLM应用于压缩文本以及多种多模态数据的有希望的结果。尽管取得了良好的性能,但仍存在一些实际问题,挑战着用LLM取代现有数据压缩算法。在本工作中,我们探索了不同的方法,以使用LLM作为数据压缩器来实现更低的调整压缩率。与先前的工作相比,我们在enwik9数据集上实现了新的最先进的(SOTA)调整压缩率,约为18%,而无需额外的模型训练。此外,我们探讨了LLM在压缩非英语数据、代码数据和字节流序列中的应用。我们证明,尽管LLM在压缩以文本为主的领域表现出色,但若配置得当,它们在压缩非自然文本序列方面仍具有竞争力。

英文摘要:

In this report, we investigate the potential use of large language models (LLM's) in the task of data compression. Previous works have demonstrated promising results in applying LLM's towards compressing not only text, but also a wide range of multi-modal data. Despite the favorable performance achieved, there still remains several practical questions that pose a challenge towards replacing existing data compression algorithms with LLM's. In this work, we explore different methods to achieve a lower adjusted compression rate using LLM's as data compressors. In comparison to previous works, we were able to achieve a new state-of-the-art (SOTA) adjusted compression rate of around $18\%$ on the enwik9 dataset without additional model training. Furthermore, we explore the use of LLM's in compressing non-English data, code data, byte stream sequences. We show that while LLM's excel in compressing data in text-dominant domains, their ability in compressing non-natural text sequences still remain competitive if configured in the right way.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑