面向制造业的事件驱动ML流水线编排：AWS行业经验
Event-Driven ML Pipeline Orchestration for Manufacturing: An AWS Industry Experience
- Liveline Technologies
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文报告了汽车制造业中事件驱动云基础设施用于持续机器学习训练的三年经验，通过ECS、SQS和Lambda实现GPU工作负载编排，实现72-78%成本降低，并验证了准入控制的必要性。
AI中文摘要：
我们呈现一份关于在汽车制造业中运行事件驱动云基础设施以进行持续机器学习训练的三年行业经验报告。我们的系统跨多个工厂编排GPU加速训练的产品专用模型对、物理预测模型和强化学习控制策略，协调由制造事件触发的长时间运行的GPU工作负载。该架构结合了Amazon ECS与EC2 GPU容量提供者、基于SQS的消息传递与死信队列，以及一个执行集群并发限制的准入控制Lambda调度器。ECS Fargate上的Conductor编排器按周计划启动依赖感知的重训练链。整个基础设施以模块化Terraform编码，并实现多账户分离。从40000多个生产训练作业中，我们报告与常开GPU基础设施相比成本降低72-78%。离散事件模拟确认准入控制是必要的（朴素调度丢失65%的作业），且队列排空与AWS Step Functions延迟匹配，同时消除了每作业启动开销。我们提供经验教训，并发布模拟器和Terraform模块骨架作为开源工件。
英文摘要：
We present an industry experience report on three years of operating an event-driven cloud infrastructure for continuous machine learning training in automotive manufacturing. Our system orchestrates GPU-accelerated training of product-specialized model pairs, a physics prediction model and a reinforcement-learning control policy, across multiple plants, coordinating long-running GPU workloads triggered by manufacturing events. The architecture combines Amazon ECS with EC2 GPU capacity providers, SQS-based messaging with dead-letter queues, and an admission-controlled Lambda dispatcher that enforces cluster concurrency limits. A Conductor orchestrator on ECS Fargate initiates dependency-aware retraining chains on a weekly schedule. The entire infrastructure is codified in modular Terraform with multi-account separation. From 40000+ production training jobs we report a 72-78% cost reduction versus always-on GPU infrastructure. A discrete-event simulation confirms that admission control is necessary (naive dispatch loses 65% of jobs) and that queue-draining matches AWS Step Functions latency while eliminating per-job startup overhead. We provide lessons learned and release the simulator and Terraform module skeletons as open-source artifacts.
