MetaCLBench: 用于资源受限边缘设备的元持续学习基准测试
MetaCLBench: Meta Continual Learning Benchmark on Resource-Constrained Edge Devices
AI总结：
MetaCLBench研究了资源受限边缘设备上元持续学习方法的准确性与部署性能，通过真实物联网设备评估六个方法，发现复杂架构未必更优，提供部署指南和框架发布。
摘要：
元持续学习（Meta-CL）使模型能够从有限的标记样本中学习新类别，使其在物联网应用中具有前景，因为手动标记成本高昂。然而，现有研究只关注准确性，而忽略了在资源受限硬件上的部署可行性。因此，我们提出了MetaCLBench，这是一个评估Meta-CL方法的基准框架，同时评估准确性和部署关键指标（内存占用、延迟和能耗）在内的真实物联网设备，其RAM大小范围从512 MB到4 GB。我们评估了六个Meta-CL方法，涵盖三种架构（CNN、YAMNet、ViT）和五个数据集，涵盖图像和音频模态。我们的评估发现，根据数据集的不同，多达六个方法中的三个在小于1 GB的设备上导致内存不足故障，显著缩小了可行的部署选项。LifeLearner在能耗方面比Oracle方法低2.54-7.43倍，同时达到接近Oracle的准确性。值得注意的是，较大的或更复杂的架构如ViT和YAMNet并不一定带来更好的Meta-CL性能，结果在不同数据集和模态中变化，挑战了关于模型复杂性的传统假设。最后，我们提供了实际部署指南，并在发表时发布我们的框架，以实现准确性和系统级指标的公平评估。
英文摘要：
Meta-Continual Learning (Meta-CL) enables models to learn new classes from limited labelled samples, making it promising for IoT applications where manual labelling is costly. However, existing studies focus on accuracy while ignoring deployment viability on resource-constrained hardware. Thus, we present MetaCLBench, a benchmark framework that evaluates Meta-CL methods for both accuracy and deployment-critical metrics (memory footprint, latency, and energy consumption) on real IoT devices with RAM sizes ranging from 512 MB to 4 GB. We evaluate six Meta-CL methods across three architectures (CNN, YAMNet, ViT) and five datasets spanning image and audio modalities. Our evaluation reveals that, depending on the dataset, up to three of six methods cause out-of-memory failures on sub-1 GB devices, significantly narrowing viable deployment options. LifeLearner achieves near-oracle accuracy while consuming 2.54-7.43x less energy than the Oracle method. Notably, larger or more sophisticated architectures such as ViT and YAMNet do not necessarily yield better Meta-CL performance, with results varying across datasets and modalities, challenging conventional assumptions about model complexity. Finally, we provide practical deployment guidelines and will release our framework upon publication to enable fair evaluation across both accuracy and system-level metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
