在新兴平台上高效部署新兴模型：一种自顶向下的测试与调试方法
Productively Deploying Emerging Models on Emerging Platforms: A Top-Down Approach for Testing and Debugging
AI总结：
针对新兴平台部署新兴模型时自底向上方法效率低下的问题，提出自顶向下的TapML方法，通过算子级测试提取和迁移策略提升测试调试效率，已成功加速105个模型部署并提供最佳实践。
摘要：
尽管现有的机器学习（ML）框架专注于成熟平台，如在服务器级 GPU 上运行 CUDA，但日益增长的需求促使我们在更广泛的场景中启用新兴 AI 应用，例如在浏览器和手机内运行大型语言模型（LLMs）。然而，由于模型快速演进以及这些平台有限的工具和实践，在新平台（如 Metal 和 WebGPU）上部署新兴模型面临着重大的软件工程挑战。以往 ML 模型部署的实践通常遵循自底向上的方式，工程师首先实现单个所需的算子，然后将它们组合起来。然而，这种传统的开发方法在部署新兴 ML 应用时无法满足生产力要求，其中测试和调试部分成为了瓶颈。为此，我们引入了 TapML，这是一种旨在简化跨多样化平台模型部署的自顶向下方法。传统的自底向上方法需要手工编写测试，而 TapML 则通过算子级测试提取自动创建高质量、真实的测试数据。此外，TapML 使用基于迁移的策略，将模型实现从成熟的源平台逐步卸载到目标平台，从而最小化复合错误的调试范围。TapML 已被用作 MLC-LLM 项目中部署新兴 ML 模型的默认开发方法。在 2 年内，TapML 加速了 5 个新兴平台上 27 种模型架构下的 105 个新兴模型的部署。我们表明，TapML 在确保已部署模型质量的同时，有效提升了开发者的生产力。此外，我们总结了来自真实世界开发过程的综合案例研究，为开发新兴 ML 系统提供了最佳实践。
英文摘要：
While existing machine learning (ML) frameworks focus on established platforms, like running CUDA on server-grade GPUs, there have been growing demands to enable emerging AI applications in a broader set of scenarios, such as running Large Language Models (LLMs) within browsers and mobile phones. However, deploying emerging models on new platforms (such as Metal and WebGPU) presents significant software engineering challenges due to rapid model evolution and limited tooling and practices for these platforms. Previous practice for ML model deployment often follows a bottom-up fashion, where engineers first implement individual required operators and then put them together. However, this traditional development approach fails to meet the productivity requirements when deploying emerging ML applications, with the testing and debugging part as a bottleneck. To this end, we introduce \textsc{TapML}, a top-down approach designed to streamline model deployment on diverse platforms. While the traditional bottom-up approach requires crafting manual tests, \textsc{TapML} automatically creates high-quality, realistic test data through operator-wise test carving. Furthermore, \textsc{TapML} uses a migration-based strategy to gradually offload model implementation from the mature source platform to the target platform, minimizing the debugging scope of compound errors. \textsc{TapML} has been used as the default development method in the MLC-LLM project to deploy emerging ML models. Within 2 years, \textsc{TapML} has accelerated the deployment of 105 emerging models in 27 model architectures across 5 emerging platforms. We show that \textsc{TapML} effectively boosts developer productivity while ensuring the quality of deployed models. Furthermore, we summarize comprehensive case studies from our real-world development, offering best practices for developing emerging ML systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
