WebLLM：一种高性能的浏览器内LLM推理引擎
WebLLM: A High-Performance In-Browser LLM Inference Engine
AI总结：
WebLLM通过JavaScript框架在浏览器内实现高性能LLM推理，利用WebGPU和WebAssembly结合MLC-LLM和Apache TVM优化，实现80%的本地性能，推动隐私保护的本地化LLM应用。
摘要：
WebLLM通过JavaScript框架在浏览器内实现高性能LLM推理，利用WebGPU和WebAssembly结合MLC-LLM和Apache TVM优化，实现80%的本地性能，推动隐私保护的本地化LLM应用。
英文摘要：
Advancements in large language models (LLMs) have unlocked remarkable capabilities. While deploying these models typically requires server-grade GPUs and cloud-based inference, the recent emergence of smaller open-source models and increasingly powerful consumer devices have made on-device deployment practical. The web browser as a platform for on-device deployment is universally accessible, provides a natural agentic environment, and conveniently abstracts out the different backends from diverse device vendors. To address this opportunity, we introduce WebLLM, an open-source JavaScript framework that enables high-performance LLM inference entirely within web browsers. WebLLM provides an OpenAI-style API for seamless integration into web applications, and leverages WebGPU for efficient local GPU acceleration and WebAssembly for performant CPU computation. With machine learning compilers MLC-LLM and Apache TVM, WebLLM leverages optimized WebGPU kernels, overcoming the absence of performant WebGPU kernel libraries. Evaluations show that WebLLM can retain up to 80% native performance on the same device, with room to further close the gap. WebLLM paves the way for universally accessible, privacy-preserving, personalized, and locally powered LLM applications in web browsers. The code is available at: https://github.com/mlc-ai/web-llm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
