Deep neural networks (DNNs) have become ubiquitous techniques in mobile and embedded systems for applications such as image/object recognition and classification. The trend of executing multiple DNNs simultaneously exacerbate the existing limitations of meeting stringent latency/accuracy requirements on resource constrained mobile devices. The prior art sheds light on exploring the accuracy-resource tradeoff by scaling the model sizes in accordance to resource dynamics. However, such model scaling approaches face to imminent challenges: (i) large space exploration of model sizes, and (ii) prohibitively long training time for different model combinations. In this paper, we present LegoDNN, a lightweight, block-grained scaling solution for running multi-DNN workloads in mobile vision systems. LegoDNN guarantees short model training times by only extracting and training a small number of common blocks (e.g. 5 in VGG and 8 in ResNet) in a DNN. At run-time, LegoDNN optimally combines the descendant models of these blocks to maximize accuracy under specific resources and latency constraints, while reducing switching overhead via smart block-level scaling of the DNN. We implement LegoDNN in TensorFlow Lite and extensively evaluate it against state-of-the-art techniques (FLOP scaling, knowledge distillation and model compression) using a set of 12 popular DNN models. Evaluation results show that LegoDNN provides 1,296x to 279,936x more options in model sizes without increasing training time, thus achieving as much as 31.74% improvement in inference accuracy and 71.07% reduction in scaling energy consumptions.
Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics
University of Waterloo(滑铁卢大学);Waterloo Artificial Intelligence Institute(滑铁卢人工智能研究所)
STeP-Diff: Spatio-Temporal Physics-Informed Diffusion Models for Mobile Fine-Grained Pollution Forecasting
Shenzhen International Graduate School, Tsinghua University, China(清华大学深圳国际研究生院);Department of Electronic Engineering, Beijing National Research Center for Information Science and Tech- nology (BNRist), Tsinghua University, China(电子工程系,北京信息科学与技术国家研究中心(BNRist),清华大学);Shenzhen Smartcity Communication, Shenzhen, China(深圳智慧城市通信);College of Soil and Water Conservation, Southwest Forestry University, China(水土保持学院,西南林业大学);College of Civil Engineering, Southwest Forestry University, China(土木工程学院,西南林业大学)
Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping
Institute for Breeding Research on Horticultural Crops, Julius Kuehn-Institute(园艺作物育种研究所,朱利叶斯·克恩研究所);Computer Graphics Group, Center for Sensor Systems (ZESS), University of Siegen(计算机图形组,传感器系统中心(ZESS),希尔德斯海姆大学)
Replacing Mobile Camera ISP with a Single Deep Learning Model
ETH Zurich(苏黎世联邦理工学院)
随着移动摄影的普及不断增长,现在正在投入大量精力来构建复杂的手工相机ISP解决方案。在这项工作中,我们证明了即使是最复杂的ISP管道也可以用一个端到端的深度学习模型来代替,而无需对特定设备中使用的传感器和光学器件有任何先验知识。为此,我们提出了PyNET,一种新型的金字塔CNN架构,专为细粒度图像复原而设计,隐式学习执行所有ISP步骤,如图像去马赛克,去噪,白平衡,颜色和对比度校正,去噪等。该模型经过训练,可以将直接从移动相机传感器获得的RAW Bayer数据转换为专业高端DSLR相机拍摄的照片,使解决方案独立于任何特定的移动ISP实现。为了在真实数据上验证所提出的方法,我们收集了一个大规模的数据集,其中包括1万个使用华为P20摄像机(12.3 MP Sony Exmor IMX 380传感器)和佳能5D Mark IV DSLR在野外拍摄的全分辨率RAW-RGB图像对。实验表明,所提出的解决方案可以很容易地达到嵌入式P20的ISP管道的水平,与我们的方法不同,它是结合来自两个(RGB + B/W)相机传感器的数据。本文中使用的数据集,预训练模型和代码可在项目网站上获得。