Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along a sequence whose tokens look identical. No analytical model states what a token costs without per-LLM fitting, and no law names the factors that set it. From closed-form expressions of LLM inference's computational and memory-access complexity, we derive the SIMPLE Inference Model to Predict LLMs Energy (SIMPLE), a four-parameter law that separates prefill from decode, and decode into KV-cache and next-token computation cost. Over traces of 13 dense LLMs from several families on NVIDIA H100 GPUs, it attains R2 = 0.969 and 11.97% MAPE, 3.1-3.5x below the SOTA, without per-LLM calibration. SIMPLE explains as well as predicts: prefill energy grows quadratically with hidden dimension, so larger LLMs suffer most from long inputs; and higher GQA KV grouping reduces the per-token energy linearly relative to standard MHA. Increasing the reasoning token budget pushes output length beyond the analytically derived energy minimum, driving super-linear energy growth; accuracy rises but energy rises faster, so Intelligence per Joule peaks at an intermediate budget and falls by up to 3.14x across a 4.0x range. Moving to agentic serving raises energy per request by 13.0x, as both prefill and KV-cache terms grow with input length; prefix caching removes most of the prefill cost (-93%), but not KV-cache cost, so even a cached agentic request costs 4.8x more.
Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency
SINTEF(辛泰夫(挪威研究机构));Singapore Management University(新加坡管理大学);Oslo Metropolitan University(奥斯陆城市大学);Kristiania University of Applied Sciences(克里斯蒂安尼亚应用科技大学)
本研究在Raspberry Pi 4上评估了28个量化LLM在能效、推理性能和准确率上的权衡,通过硬件级能耗分析与多数据集基准测试,为资源受限环境下的可持续LLM部署提供了可操作的见解。
Evaluating the Correctness of Inference Patterns Used by LLMs for Judgment
Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);Beijing Institute for General Artificial Intelligence(北京一般人工智能研究所)
Not All Layers of LLMs Are Necessary During Inference
University of Electronic Science and Technology of China(电子科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机学院)
Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
School of Cyber Science and Engineering, Xi’an Jiaotong University(西安交通大学计算机科学与工程学院);School of Computer Science and Engineering, The University of Aizu(台湾花莲大学计算机科学与工程学院)
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families
Department of Statistics, University of Michigan(密歇根大学统计学系);MIT-IBM Watson AI Lab, IBM Research(MIT-IBM Watson AI实验室,IBM研究);Computer Science and Artificial Intelligence Laboratory, MIT(MIT计算机科学与人工智能实验室);Institute of Foundation Models, MBZUAI(基础模型研究所,MBZUAI)
Online Scheduling for LLM Inference with KV Cache Constraints
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology(麻省理工学院电子工程与计算机科学系);HKUST(香港科技大学);Microsoft Research(微软研究院);Sloan School of Management, Massachusetts Institute of Technology(斯隆管理学院,麻省理工学院);Operations Research Center, Massachusetts Institute of Technology(运营研究中心,麻省理工学院)
When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning
Institute of Science Tokyo(东京科学大学);Osaka University(大阪大学);NII LLMC(日本信息学研究所语言、语言学与媒体中心);Nara Institute of Science and Technology(奈良科学技术研究所);Center of Juris-Informatics, ROIS-DS(日本学术研究推进机构跨学科科学研究中心法学信息学中心)
Humans and LLMs Diverge on Probabilistic Inferences
McGill University(麦吉尔大学);Mila – Quebec AI Institute(魁北克人工智能研究所);University of Vienna(维也纳大学);FNRS – UCLouvain(佛罗伦萨-鲁汶大学);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);School of Computer Science and Engineering, Beihang University(北航计算机科学与工程学院)
Cross-LLM Consistency in Inference: Evidence from Shared Interactions
School of Computer Science Shanghai Jiao Tong University(上海交通大学计算机科学学院);Ningbo Key Laboratory of Advanced Manufacturing Simulation Eastern Institute of Technology, Ningbo(宁波市先进制造仿真重点实验室,宁波东方理工大学);College of Computer and Information Science Chongqing Normal University(重庆师范大学计算机与信息科学学院);SymtrustAI.com;Eastern Institute of Technology, Ningbo(宁波东方理工大学)