GEMEL：边缘模型合并，实现内存高效、实时的视频分析
GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge
摘要：
视频分析管道已稳步转向边缘部署，以减少带宽开销和隐私侵犯，但在此过程中，面临着不断增长的资源紧张。最值得注意的是，边缘盒GPU缺乏同时容纳越来越多（越来越复杂）的实时的【形】推理模型所需的内存。不幸的是，依赖于GPU资源的时间/空间共享的现有解决方案是不够的，因为所需的交换延迟导致不可接受的帧丢失和精度违规。我们提出了模型合并，一种新的存储管理技术，利用边缘视觉模型之间的架构相似性，明智地共享他们的层（包括权重），以减少工作负载的内存成本和交换延迟。我们的系统GEMEL通过以下方式将合并有效地集成到现有管道中：（1）利用关于每个模型内存使用和层间依赖关系的几个指导性观察，以快速识别富有成效和保持准确性的合并配置，以及（2）改变边缘推理计划以最大限度地提高合并效益。在不同工作负载上的实验表明，相对于单独的时间/空间共享，GEMEL减少了高达60.7%的内存使用，并将整体准确性提高了8-39%。
英文摘要：
Video analytics pipelines have steadily shifted to edge deployments to reduce bandwidth overheads and privacy violations, but in doing so, face an ever-growing resource tension. Most notably, edge-box GPUs lack the memory needed to concurrently house the growing number of (increasingly complex) models for real-time inference. Unfortunately, existing solutions that rely on time/space sharing of GPU resources are insufficient as the required swapping delays result in unacceptable frame drops and accuracy violations. We present model merging, a new memory management technique that exploits architectural similarities between edge vision models by judiciously sharing their layers (including weights) to reduce workload memory costs and swapping delays. Our system, GEMEL, efficiently integrates merging into existing pipelines by (1) leveraging several guiding observations about per-model memory usage and inter-layer dependencies to quickly identify fruitful and accuracy-preserving merging configurations, and (2) altering edge inference schedules to maximize merging benefits. Experiments across diverse workloads reveal that GEMEL reduces memory usage by up to 60.7%, and improves overall accuracy by 8-39% relative to time/space sharing alone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
