Large language models (LLMs) have enabled time series (TS) analysis by jointly modeling numerical observations and textual context through a shared token interface. However, TS tokens and prompt tokens exhibit fundamentally different information structures, making uniform token processing inefficient. In this paper, we study token efficiency in TS language modeling from an asymmetric-token perspective. We show that TS tokens have highly uneven spectral contributions, where many tokens share redundant frequency patterns while a small subset preserves critical temporal evidence. We also observe that prompt-token influence attenuates with model depth, suggesting that full prompt retention across all layers is unnecessary. Based on these findings, we develop an adaptive token budgeting framework that compresses TS tokens via frequency-domain structure and progressively reduces prompt tokens across layers. Experiments across forecasting, classification, imputation, and anomaly detection demonstrate up to \textit{\textbf{7.68$\times$}} inference acceleration and performance gains in \textit{\textbf{78\%}} of evaluated settings, showing the effectiveness of asymmetric token compression for scalable TS foundation models.
OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences (CASIA)(模式识别新实验室(NLPR)、自动化研究所、中国科学院(CASIA));Nanjing University(南京大学);The Hong Kong University of Science(香港科学大学);Sichuan University(四川大学);Peking University(北京大学)
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院自然语言处理实验室);NiuTrans Research, Shenyang, China(NiuTrans研究院)
Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院,中国沈阳);NiuTrans Research, Shenyang, China(沈阳NiuTrans研究院,中国);Kunming University of Science and Technology, Kunming, China(昆明理工大学,中国昆明)
TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction
Meetyou AI Lab(美柚人工智能实验室);Xiamen Key Laboratory of Women’s Internet Health Management(厦门市女性互联网健康管理重点实验室);Osaka University(大阪大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局)
TREX: Tokenizer Regression for Optimal Data Mixture
KAIST CT(韩国科学技术院CT);Seoul National University of Science and Technology(首尔科学技术大学);Upstage AI;KAIST InnoCORE PRISM-AI Center(韩国科学技术院InnoCORE PRISM-AI中心)
MultiTok: Variable-Length Tokenization for Efficient LLMs Adapted from LZW Compression
University of Texas at Austin(德克萨斯大学奥斯汀分校);Nokia Bell Labs(诺基亚贝尔实验室);Boğaziçi University(博雅大学);Georgia Institute of Technology(佐治亚理工学院);Massachusetts Institute of Technology (MIT)(麻省理工学院(MIT))
Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines
VIA Cybersecurity Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana(维亚网络安全实验室,卡布雷姆大学科学与技术学院,库马西,加纳);Quantum and Assistive Technologies Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana(量子与辅助技术实验室,卡布雷姆大学科学与技术学院,库马西,加纳)
Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Shenzhen Loop Area Institute(深圳河套学院);University of Chinese Academy of Sciences(中国科学院大学);The Hong Kong University of Science and Technology(香港科技大学)
UIPress: Bringing Optical Token Compression to UI-to-Code Generation
The Chinese University of Hong Kong(香港中文大学);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学);University of Technology Sydney(悉尼科技大学)