Autoregressive large language models (LLMs) are bottlenecked by sequential decoding, where each new token typically requires executing all transformer layers. Existing dynamic-depth and layer-skipping methods reduce this cost, but often rely on auxiliary routing mechanisms or incur accuracy degradation when bypassed layers are left uncompensated. We present \textbf{LoRA-Drop}, a plug-and-play inference framework that accelerates decoding by applying a \emph{temporal compute schedule} to a fixed subset of intermediate layers: on most decoding steps, selected layers reuse the previous-token hidden state and apply a low-rank LoRA correction, while periodic \emph{refresh} steps execute the full model to prevent drift. LoRA-Drop requires no routing network, is compatible with standard KV caching, and can reduce KV-cache footprint by skipping KV updates in droppable layers during LoRA steps and refreshing periodically. Across \textbf{LLaMA2-7B}, \textbf{LLaMA3-8B}, \textbf{Qwen2.5-7B}, and \textbf{Qwen2.5-14B}, LoRA-Drop achieves up to \textbf{2.6$\times$ faster decoding} and \textbf{45--55\% KV-cache reduction} while staying within \textbf{0.5 percentage points (pp)} of baseline accuracy. Evaluations on reasoning (GSM8K, MATH, BBH), code generation (HumanEval, MBPP), and long-context/multilingual benchmarks (LongBench, XNLI, XCOPA) identify a consistent \emph{safe zone} of scheduling configurations that preserves quality while delivering substantial efficiency gains, providing a simple path toward adaptive-capacity inference in LLMs. Codes are available at https://github.com/hosseinbv/LoRA-Drop.git.
They Said Memes Were Harmless-We Found the Ones That Hurt: Decoding Jokes, Symbols, and Cultural References
Macquarie University（麦考瑞大学）；The University of Western Australia（西澳大学）；Stanford University（斯坦福大学）；Institute for Clarity in Documentation（文档清晰研究所）；Inria Paris-Rocquencourt（巴黎-罗quentcourt研究所）；Rajiv Gandhi University（拉贾·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
FAIR at Meta（Meta FAIR研究院）；GenAI at Meta（Meta GenAI部门）；Reality Labs at Meta（Meta Reality Labs）；University of Toronto（多伦多大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Dana-Farber Cancer Institute（丹娜法伯癌症研究院）；Carnegie Mellon University（卡内基梅隆大学）
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
Peking University（北京大学）；XiaoMi AI Lab（小米AI实验室）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；University of Electronic Science and Technology of China（电子科技大学）
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Eindhoven University of Technology（埃因霍温理工大学）；University of Aberdeen（阿伯丁大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Oxford（牛津大学）
ClaimIQ at CheckThat! 2025: Comparing Prompted and Fine-Tuned Language Models for Verifying Numerical Claims
Department of Data Science, University of North Texas, Denton, TX, USA（数据科学系，北卡罗来纳州立大学，德顿，得克萨斯州，美国）；Department of Computer Science and Engineering, University of North Texas, Denton, TX, USA（计算机科学与工程系，北卡罗来纳州立大学，德顿，得克萨斯州，美国）；Department of Computer Science（计算机科学系）
