arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.26470cs.LG
收藏

边缘端实时动态推断的分层自适应控制

Hierarchical adaptive control for real-time dynamic inference at the edge

MMMI Software Engineering(MMMI软件工程)

Francesco Daghero, Mahyar Tourchi Moghaddam, Mikkel Baun Kjærgaard

AI总结:

本文提出分层自适应架构,通过优化模型与系统决策,降低边缘节点的推理延迟和能耗,同时保持高效率和准确性。

摘要:

工业系统日益依赖机器学习(ML),其在异构节点上运行需满足严格的延迟、能耗和内存约束。动态ML模型在运行时重新配置计算足迹,可提供高能效和较低的平均延迟,但部署复杂,因其依赖额外的超参数。这些超参数控制精度与平均延迟的权衡,通常在校准数据集上调整,该数据集必须匹配测试时间分布,这一假设在现实场景中很少成立,导致操作条件不理想,可能低于静态模型。本文提出双层自适应架构,共同优化模型和系统决策。在全局层面,调度器为每个边缘节点配置并部署由轻量专用模型和通用fallback组成的分类器级联,满足延迟和内存约束。在节点层面,本地控制器跟踪数据漂移和硬件资源,启用或禁用专用预测器(SP)以保持高能效并避免延迟约束违规。此设计允许更长的运行时间而不强制全局重新部署,且在远程全局控制器不可达时仍能高效执行。我们在两个数据集上评估该方法,在受控分布不匹配场景下,显示平均推理延迟减少高达2.45倍,能耗减少高达2.86倍,与静态基线相比精度下降低于4%。我们的贡献包括:(1)一种预算化的SP级联公式,保持最坏情况延迟约束;(2)一种分层控制器,保持数据和资源变化下的效率;(3)在嵌入式硬件上的实验评估。

英文摘要:

Industrial systems increasingly depend on Machine Learning (ML), and operate on heterogeneous nodes that must satisfy tight latency, energy, and memory constraints. Dynamic ML models, which reconfigure their computational footprint at runtime, promise high energy efficiency and lower average latency for modest accuracy tradeoffs; however, their deployment is complex due to the additional hyperparameters they rely on. These hyperparameters, controlling the accuracy versus average latency tradeoff, are often tuned on a calibration dataset that must match the test time distribution, an assumption that rarely holds in real-world scenarios, leading to suboptimal operational conditions, possibly below static models. We propose a two-tier adaptive architecture that co-optimizes model and system decisions. At the global level, a scheduler configures and deploys, for each edge node, a cascade of classifiers composed of lightweight specialized models and a generalist fallback, satisfying latency and memory constraints. At the node level, a local controller tracks data drifts and hardware resources, enabling or disabling specialized predictors (SP) to preserve high energy efficiency and avoid latency-constraint violations under varying conditions. This design allows longer operating times without forcing a global redeployment step, and enables efficient execution in case of an unreachable remote global controller. We evaluate the approach on two datasets under controlled distribution mismatch scenarios, showing average per-inference reductions of latency up to 2.45x and energy up to 2.86x, with less than 4% accuracy drop compared to static baselines. Our contributions are:(1) a budgeted SP-cascade formulation that preserves worst-case latency constraints;(2) a hierarchical controller that maintains efficiency under data and resource changes; and (3) an experimental evaluation on embedded hardware.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑