arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1812.02240cs.LGstat.ML
收藏

面向真实世界系统的高效鲁棒机器学习

Efficient and Robust Machine Learning for Real-World Systems

Graz University of Technology(格拉茨工业大学);Ruperts Karls University(鲁珀特卡尔斯大学);Microsoft Research(微软研究院);University of Cambridge(剑桥大学);Uber AI Labs(优步人工智能实验室);Arm Research(Arm研究院);Arm Ltd.(Arm有限公司)

Franz Pernkopf, Wolfgang Roth, Matthias Zoehrer, Lukas Pfeifenberger, Guenther Schindler, Holger Froening, Sebastian Tschiatschek, Robert Peharz, Matthew Mattina, Zoubin Ghahramani

摘要:

虽然机器学习传统上是一项资源密集型任务,但嵌入式系统、自主导航和物联网的愿景激发了人们对资源高效方法的兴趣。这些方法需要在计算和能量方面的性能和资源消耗之间仔细选择权衡。除此之外,在机器学习系统的所有应用中,以一致的方式处理不确定性至关重要。特别是,任何现实世界的系统都需要在存在异常值和损坏数据的情况下保持鲁棒性,并且“意识到”其局限性,即该系统应当保持并提供关于其自身预测的不确定性估计。这些复杂的需求是当前机器学习研究的主要挑战之一,也是确保机器学习技术顺利过渡到日常应用的关键。在这篇文章中,我们提供了一个概述的机器学习技术促进这些现实世界的需求的最新发展。首先,我们对深度神经网络中的资源效率进行了全面的回顾,重点关注模型大小缩减、压缩和精度降低的技术。这些技术可以在训练期间或作为后处理应用,并且被广泛用于降低计算复杂性和存储器占用。由于大多数(实用)神经网络在处理不确定性的方式上都有局限性,我们将其与概率图模型进行了对比,后者通过概率推理很容易满足这些需求。通过这种方式,我们提供了一个广泛的概述,目前的最先进的强大和有效的机器学习的现实世界的系统。

英文摘要:

While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation and the vision of the Internet-of-Things fuel the interest in resource efficient approaches. These approaches require a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. On top of this, it is crucial to treat uncertainty in a consistent manner in all but the simplest applications of machine learning systems. In particular, a desideratum for any real-world system is to be robust in the presence of outliers and corrupted data, as well as being `aware' of its limits, i.e.\ the system should maintain and provide an uncertainty estimate over its own predictions. These complex demands are among the major challenges in current machine learning research and key to ensure a smooth transition of machine learning technology into every day's applications. In this article, we provide an overview of the current state of the art of machine learning techniques facilitating these real-world requirements. First we provide a comprehensive review of resource-efficiency in deep neural networks with focus on techniques for model size reduction, compression and reduced precision. These techniques can be applied during training or as post-processing and are widely used to reduce both computational complexity and memory footprint. As most (practical) neural networks are limited in their ways to treat uncertainty, we contrast them with probabilistic graphical models, which readily serve these desiderata by means of probabilistic inference. In that way, we provide an extensive overview of the current state-of-the-art of robust and efficient machine learning for real-world systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑