Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures. However, existing studies evaluate these methods in narrow settings, leaving unclear whether tensorization is effective at large-scale deployment. We systematically evaluate tensor compression across dense and MoE architectures, establishing performance trade-offs grounded in both empirical analysis and theoretical analysis. We identify a fundamental mismatch between the shared subspaces assumed by tensor decompositions and the heterogeneous representations learned by modern LLMs, thereby delineating their practical limits and clarifying their viable role in large-scale deployment. The code is available at https://github.com/brain-lab-research/TT-LLM.
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National University of Singapore（新加坡国立大学）
Rethinking Soft Compression in Retrieval-Augmented Generation: A Query-Conditioned Selector Perspective
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence（上海数据科学 key 实验室，计算机科学与人工智能学院）；Fudan University（复旦大学）；Shanghai（上海）；China（中国）
VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Honor Device Co., Ltd（荣耀设备有限公司）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身人工智能研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海多模态具身人工智能重点实验室）
Test of Time: Rethinking Temporal Signal of Benchmark Contamination
Jinesis Lab, University of Toronto & Vector Institute（Jinesis实验室、多伦多大学及向量研究所）；ETH Zürich & ETH AI Center（苏黎世联邦理工学院及ETH人工智能中心）；Max Planck Institute for Intelligent Systems, Tübingen, Germany（智能系统马克斯·普朗克研究所，图宾根，德国）；ELLIS Institute Tübingen（图宾根ELLIS研究所）
Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西湖大学工程学院）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖先进研究院技术研究所）；Huawei Noah’s Ark Lab（华为诺亚实验室）
No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
Zhongguancun Academy（中关村学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
IST Austria（奥地利科学技术研究所）；ETH Zurich（苏黎世联邦理工学院）；Neural Magic（神经魔法公司）
We consider the problem of model compression for deep neural networks (DNNs) in the challenging one-shot/post-training setting, in which we are given an accurate trained model, and must compress it without any retraining, based only on a small amount of calibration input data. This problem has become popular in view of the emerging software and hardware support for executing models compressed via pruning and/or quantization with speedup, and well-performing solutions have been proposed independently for both compression approaches. In this paper, we introduce a new compression framework which …
University of Rochester（罗切斯特大学）；Tencent AI Lab（腾讯人工智能实验室）；University of Pennsylvania（宾夕法尼亚大学）
Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the dec…
Rethinking Token Reduction for Large Vision-Language Models
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；National University of Singapore（新加坡国立大学）；Alibaba Group（阿里巴巴集团）
EfficientPosterGen: Semantic-aware Efficient Poster Generation via Token Compression and Accurate Violation Detection
Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Harbin Institute of Technology（哈尔滨工业大学）；Wuhan University（武汉大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Dalian Maritime University（大连海事大学）；Renmin University of China（中国人民大学）
Model compression and acceleration are attracting increasing attentions due to the demand for embedded devices and mobile applications. Research on efficient convolutional neural networks (CNNs) aims at removing feature redundancy by decomposing or optimizing the convolutional calculation. In this work, feature redundancy is assumed to exist among channels in CNN architectures, which provides some leeway to boost calculation efficiency. Aiming at channel compression, a novel convolutional construction named compact convolution is proposed to embrace the progress in spatial convolution, channel…
Rethinking Parameter Sharing as Graph Coloring for Structured Compression
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is not a synonym for "high perceptual quality", and in fact optimization of one often comes at the expense of the other. In light of this understanding, it is natural to seek for a generalization of rate-distortion theory which takes perceptual quality into account. In this paper, we …
Rethinking the Potential of Layer Freezing for Efficient DNN Training
University of Georgia（佐治亚大学）；Northeastern University（东北大学）；University of Pittsburgh（匹兹堡大学）；The University of Arizona（亚利桑那大学）；Stevens Institute of Technology（斯蒂文斯理工学院）；The University of North Carolina at Charlotte（北卡罗来纳大学教堂山分校）；University of Virginia（弗吉尼亚大学）
Biologically inspired spiking neural networks (SNNs) are widely used to realize ultralow-power energy consumption. However, deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers. To tackle this problem, we propose a novel but simple normalization technique called postsynaptic potential normalization. This normalization removes the subtraction term from the standard normalization and uses the second raw moment instead of the variance as the division term. The spike firing can be controlled, enabling the training to proceed appropriating, by conducti…
Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond
Columbia University（哥伦比亚大学）；Department of Computer Science, New York University（纽约大学计算机科学系）；Microsoft Research（微软研究院）；Center for Data Science, New York University（纽约大学数据科学中心）
Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
Department of Data Science and AI（数据科学与人工智能系）；Monash University（墨尔本大学）；Centre for Vision, Speech and Signal Processing（视觉、语音和信号处理中心）；University of Surrey（萨里大学）
Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample Construction
