arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.22261cs.LG
收藏

利用上下文感知的模型切换实现可持续的LLM推理

Sustainable LLM Inference using Context-Aware Model Switching

Faculty of Computer Science and Information Technology, Universiti Malaya(马来亚大学计算机科学与信息技术学院)

Yuvarani, Akashdeep Singh, Zahra Fathanah, Salsabila Harlen, Syeikha Syafura Al-Zahra binti Zahari, Hema Subramaniam

AI总结:

本文提出一种基于上下文感知的模型切换方法,通过动态选择适合的模型来降低LLM推理的能量消耗,实验表明可减少67.5%的能耗并保持高质量响应。

摘要:

大型语言模型已成为许多AI应用的核心,但其日益增长的能量消耗引发了严重的可持续性问题。当前AI部署中的一个关键限制是依赖于

英文摘要:

Large language models have become central to many AI applications, but their growing energy consumption raises serious sustainability concerns. A key limitation in current AI deployments is the reliance on a one-size-fits-all inference strategy where most systems route every request to the same large model, regardless of task complexity, leading to substantial and unnecessary energy waste. To address this issue, we propose a context-aware model switching approach that dynamically selects an appropriate language model based on query complexity. The proposed system uses a Context-Aware Model Switching for Energy-Efficient LLM Inference that combines caching for repeated queries, rulebased complexity scoring for fast and explainable decisions, machine learning classification to capture semantic intent, and a user-adaptive component that learns from interaction patterns over time. The proposed architecture was evaluated using real conversation workloads and three open-source language models (Gemma3 1B, Gemma3 4B and Qwen3 4B) with different computational costs, measuring energy consumption (via NVML GPU power telemetry), response latency, routing accuracy, and output quality (BERTScore F1) to reflect real-world usage conditions. Experimental results show that the model switching approach can reduce energy consumption by up to 67.5% compared to always using the largest model while maintaining a response quality of 93.6%. In addition, the response time for simple queries also improved significantly by approximately 68%. These results show that model switching inference offers a practical and scalable path toward more energy-efficient and sustainable AI systems, demonstrating that significant efficiency gains can be achieved without major sacrifices in response quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑