EdgeAI（基于边缘计算的人工智能）在过去几年中一直是最活跃的研究，以处理各种大规模分布式AI应用程序，以满足严格的延迟要求。与此同时，许多公司已经发布了具有较小形状因子（低功耗和有限资源）的边缘设备，如流行的Raspberry Pi和Nvidia的Jetson Nano，用于在边缘计算环境中充当计算节点。虽然边缘设备在计算能力和硬件资源方面受到限制，但它们由加速器提供动力，以增强其性能行为。因此，看看基于AI的深度神经网络如何在资源有限的设备上运行是很有趣的。在这项工作中，我们提出并比较了四个片上系统（SoC）的推理时间和功耗方面的性能：华硕Tinker Edge R，Raspberry Pi 4，Google Coral Dev Board，Nvidia Jetson Nano和一个微控制器：Arduino Nano 33 BLE，在不同的深度学习模型和框架上。我们还提供了一种方法来测量功耗，推理时间和准确性的设备，它可以很容易地扩展到其他设备。我们的结果表明，对于基于Tensorflow的量化模型，Google Coral Dev Board在推理时间和功耗方面都提供了最佳性能。对于推理计算时间的一小部分，即MobileNetV2的时间不到29.3%，Jetson Nano比其他设备执行得更快。
英文摘要：
EdgeAI (Edge computing based Artificial Intelligence) has been most actively researched for the last few years to handle variety of massively distributed AI applications to meet up the strict latency requirements. Meanwhile, many companies have released edge devices with smaller form factors (low power consumption and limited resources) like the popular Raspberry Pi and Nvidia's Jetson Nano for acting as compute nodes at the edge computing environments. Although the edge devices are limited in terms of computing power and hardware resources, they are powered by accelerators to enhance their performance behavior. Therefore, it is interesting to see how AI-based Deep Neural Networks perform on such devices with limited resources.
In this work, we present and compare the performance in terms of inference time and power consumption of the four Systems on a Chip (SoCs): Asus Tinker Edge R, Raspberry Pi 4, Google Coral Dev Board, Nvidia Jetson Nano, and one microcontroller: Arduino Nano 33 BLE, on different deep learning models and frameworks. We also provide a method for measuring power consumption, inference time and accuracy for the devices, which can be easily extended to other devices. Our results showcase that, for Tensorflow based quantized model, the Google Coral Dev Board delivers the best performance, both for inference time and power consumption. For a low fraction of inference computation time, i.e. less than 29.3% of the time for MobileNetV2, the Jetson Nano performs faster than the other devices.
Benchmarking of CPU-intensive Stream Data Processing in The Edge Computing Systems
AGH University of Science and Technology, Poland（波兰AGH科学技术大学）；Newcastle University, UK（英国新castle大学）；Newcastle University Newcastle upon Tyne, UK（英国新castle大学新castle upon Tyne分校）；IBM TJ Watson Yorktown Heights, NY, USA（美国IBM TJ沃森实验室）
Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computers
Kaze Technologies（凯兹技术）；Kaze Consulting（凯兹咨询）；Electrical and Computer Engineering（电气与计算机工程）；Lebanese American University（黎巴嫩美国大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Exeter（埃克塞特大学）
Benchmarking and Adapting On-Device LLMs for Clinical Decision Support
AI Collaborative Centre, University Health Network（人工智能协同中心，健康大学网络）；Princess Margaret Cancer Centre, University Health Network（彭菲癌症中心，健康大学网络）；Department of Electrical and Computer Engineering, University of Toronto（电气与计算机工程系，多伦多大学）；Division of Urology, Department of Surgery, St. Michael’s Hospital, Unity Health Toronto and University of Toronto（泌尿科，外科部，圣米歇尔医院，统一健康多伦多及多伦多大学）；Peter Munk Cardiac Centre, University Health Network（彼得·默克心脏中心，健康大学网络）；Department of Laboratory Medicine and Pathobiology and Department of Computer Science, University of Toronto（实验室医学与病理学系及计算机科学系，多伦多大学）；Vector Institute, Toronto（向量研究所，多伦多）
Henan Key Laboratory of Quantum Information and Cryptography（河南省量子信息与密码学重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Quantum Information Sciences（北京量子信息科学研究院）；Tsinghua University（清华大学）
Distributed Threat Intelligence at the Edge Devices: A Large Language Model-Driven Approach
Southern Illinois University（南伊利诺伊大学）；School of Computing（计算机学院）；Center for Research and Education in AI and Cybersecurity (CARE-AI-C)（人工智能与网络安全研究教育中心）
