arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.09175cs.LG
收藏

无记忆连续深度神经网络中有效防止角距离语义漂移

Effective prevention of semantic drift as angular distance in memory-less continual deep neural networks

National University of Computer and Emerging Sciences(国立计算机与新兴科学大学)

Khouloud Saadi, Muhammad Taimoor Khan

摘要:

终身机器学习或持续学习模型试图通过在一系列任务中积累知识来逐步学习。因此,这些模型学习得更好、更快。它们用于必须与人类或任何动态环境交互的各种智能系统中,聊天机器人和自动驾驶汽车。无记忆方法更常用于深度神经网络,该网络在其架构内容纳来自任务的传入信息。它允许他们在所有可见的任务上表现良好。这些模型遭受语义漂移或可塑性稳定性困境。现有的模型使用Minkowski距离度量来决定冻结、更新或复制哪些节点。这些距离度量不提供更好的节点分离,因为它们易受高维稀疏向量的影响。在我们提出的方法中,我们使用角距离来评估单个节点中的语义漂移,从而提供更好的节点分离,从而更好地平衡稳定性和可塑性。所提出的方法通过在标准数据集上保持更高的准确性而优于最先进的模型。

英文摘要:

Lifelong machine learning or continual learning models attempt to learn incrementally by accumulating knowledge across a sequence of tasks. Therefore, these models learn better and faster. They are used in various intelligent systems that have to interact with humans or any dynamic environment e.g., chatbots and self-driving cars. Memory-less approach is more often used with deep neural networks that accommodates incoming information from tasks within its architecture. It allows them to perform well on all the seen tasks. These models suffer from semantic drift or the plasticity-stability dilemma. The existing models use Minkowski distance measures to decide which nodes to freeze, update or duplicate. These distance metrics do not provide better separation of nodes as they are susceptible to high dimensional sparse vectors. In our proposed approach, we use angular distance to evaluate the semantic drift in individual nodes that provide better separation of nodes and thus better balancing between stability and plasticity. The proposed approach outperforms state-of-the art models by maintaining higher accuracy on standard datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑