HyperEmbed:基于超维计算的NLP任务中资源与性能的权衡
HyperEmbed: Tradeoffs Between Resources and Performance in NLP Tasks with Hyperdimensional Computing enabled Embedding of n-gram Statistics
摘要:
深度学习的最新进展已经导致几个NLP任务的性能显着提高,然而,模型的计算要求越来越高。因此,本文处理域的计算效率算法的NLP任务。特别是,它研究了文本的n-gram统计的分布式表示。使用支持超维计算的嵌入来形成表示。然后,这些表示作为特征,这些特征被用作标准分类器的输入。我们调查的适用性嵌入一个大的和三个小的标准数据集的分类任务,使用九个分类器。与传统的n-gram统计相比,在F1分数上实现的嵌入同时将时间和内存要求降低了数倍,例如,对于小数据集上的分类器之一,存储器减少为6.18倍;而训练和测试加速分别为4.62和3.84倍。对于大型数据集上的许多分类器,内存减少是ca。训练和试验提速100多倍。重要的是,通过超维计算形成的分布式表示的使用允许剖析表示的维度和n-gram大小之间的严格依赖性,从而为权衡打开了空间。
英文摘要:
Recent advances in Deep Learning have led to a significant performance increase on several NLP tasks, however, the models become more and more computationally demanding. Therefore, this paper tackles the domain of computationally efficient algorithms for NLP tasks. In particular, it investigates distributed representations of n-gram statistics of texts. The representations are formed using hyperdimensional computing enabled embedding. These representations then serve as features, which are used as input to standard classifiers. We investigate the applicability of the embedding on one large and three small standard datasets for classification tasks using nine classifiers. The embedding achieved on par F1 scores while decreasing the time and memory requirements by several times compared to the conventional n-gram statistics, e.g., for one of the classifiers on a small dataset, the memory reduction was 6.18 times; while train and test speed-ups were 4.62 and 3.84 times, respectively. For many classifiers on the large dataset, memory reduction was ca. 100 times and train and test speed-ups were over 100 times. Importantly, the usage of distributed representations formed via hyperdimensional computing allows dissecting strict dependency between the dimensionality of the representation and n-gram size, thus, opening a room for tradeoffs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。