通过统一少样本、零样本、持续和上下文学习实现边缘端通用设备适配
Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning
浏览论文内容
中文总结 AI 辅助
本研究提出以嵌入器为中心的学习框架，统一四类在线学习场景，在硬件上实现资源受限设备的通用适配，在多项任务中达到SOTA性能，无需依赖云端即可完成边缘端个性化学习。
中文摘要 AI 辅助
随着智能边缘设备的日益普及，针对用户（如自定义关键词检测）或患者（如自适应健康监测）的定制化应用需求不断增长。然而，大多数边缘设备依赖固定推理算法，无法在设备端学习以个性化预测；即便支持学习，通常也仅适配特定学习场景，例如少样本学习（FSL），若需扩展至其他场景，则需借助专用设备或云端重新训练，这会带来显著的能耗与延迟开销、缺乏实时能力及隐私问题。本研究提出以嵌入器为中心的学习（ECL）框架，统一四类在线学习场景：用于即时定制的少样本学习（FSL）、用于知识积累的持续学习（CL）、用于利用语义数据的零样本学习（ZSL），以及用于超越分类的上下文学习（ICL）。我们在硬件上验证，ECL可部署于资源受限设备，覆盖上述学习场景的四类真实用例。该方法在FSL字符识别任务中达到新的SOTA性能（Omniglot数据集：5-way 1-shot准确率96.8%，32-way 1-shot准确率83.3%），并在关键词检测任务中建立CL的首个硬件基线（NeuroBench关键词FSCIL任务：200-way 5-shot准确率71.8%）；此外，我们还展示了ZSL（利用语义数据的5-way语音句子分类，准确率60.6%）和ICL（RegBench任务第500个token处准确率46.2%）的首个硬件演示，所有操作均在微瓦至毫瓦级功耗预算下运行。因此，通过统一多类学习场景，我们为无需依赖云端、可在边缘端直接适配的智能通用设备铺平了道路。
英文摘要
With the ever-increasing pervasiveness of smart edge devices, the demand is growing for applications that can be tailored to users (e.g., custom keyword spotting) or patients (e.g., adaptive health monitoring). Yet, most edge devices rely on fixed inference algorithms and thus cannot learn on-device to personalize predictions. When they can, devices typically support only a specific learning scenario, such as few-shot learning (FSL): going beyond this requires resorting either to another specialized device or to cloud-based retraining, which implies significant energy and latency overheads, a lack of real-time capabilities, and privacy concerns. In this work, we introduce embedder-centric learning (ECL), a framework that unifies four different online learning scenarios: FSL for on-the-fly customization, continual learning (CL) for knowledge accumulation, zero-shot learning (ZSL) for leveraging semantic data, and in-context learning (ICL) for adapting beyond classification. We demonstrate in silicon that ECL can be deployed on resource-constrained devices across four real-world use cases representative of the aforementioned learning scenarios. Our approach establishes a new state-of-the-art performance for FSL character recognition (Omniglot: 96.8% for 5-way 1-shot, 83.3% for 32-way 1-shot), and the first hardware baseline for CL in keyword spotting (NeuroBench keyword FSCIL: 71.8% for 200-way 5-shot). Moreover, we present the first hardware demonstrations of ZSL with semantic data (60.6% for 5-way spoken sentence classification) and ICL (46.2% at the 500th token of RegBench) operating at micro-to-milliwatt power budgets. Therefore, by unifying multiple learning scenarios, we pave the way for smart and versatile devices that can adapt right at the edge, without reliance on the cloud.
发表机构
- Delft University of Technology（代尔夫特理工大学）
机构由 AI 辅助整理，请以论文原文为准。
