arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1709.01547cs.AI
收藏

人工智能系统间的知识传递

Knowledge Transfer Between Artificial Intelligence Systems

University of Leicester(莱斯特大学);St. Petersburg State University of Electrical Engineering(圣彼得堡国立电气工程大学);UnivAI Ltd(UnivAI公司);ARM Ltd(ARM公司)

Ivan Y. Tyukin, Alexander N. Gorban, Konstantin Sofeikov, Ilya Romanenko

摘要:

我们考虑的基本问题是:传统的“学生”人工智能(AI)系统如何从传统的“教师”AI系统或人类专家那里学习,而不需要完全的重新训练,最重要的是,不需要大量的计算资源。在这里,“学习”被理解为一个系统模仿另一个系统的反应的能力,反之亦然。我们称这种学习为人工智能知识转移。我们表明,如果内部变量的“学生”人工智能系统的结构的$n$维拓扑向量空间和$n$是足够高的概率接近1,所需的知识转移可以通过简单的级联的线性泛函。特别地,对于足够大的$n$,概率接近1,“学生”系统可以以两个额外的内积为代价成功地且非迭代地从“教师”学习$k\ll n$新示例(或纠正相同数量的错误)。这个概念是用一个例子来说明的知识转移从一个预先训练的卷积神经网络一个简单的线性分类器与HOG功能。

英文摘要:

We consider the fundamental question: how a legacy "student" Artificial Intelligent (AI) system could learn from a legacy "teacher" AI system or a human expert without complete re-training and, most importantly, without requiring significant computational resources. Here "learning" is understood as an ability of one system to mimic responses of the other and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the "student" Artificial Intelligent system have the structure of an $n$-dimensional topological vector space and $n$ is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for $n$ sufficiently large, with probability close to one, the "student" system can successfully and non-iteratively learn $k\ll n$ new examples from the "teacher" (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from a pre-trained convolutional neural network to a simple linear classifier with HOG features.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑