一种新的多步确定性Tsetlin机器学习自动机
A Novel Multi-Step Finite-State Automaton for Arbitrarily Deterministic Tsetlin Machine Learning
摘要:
由于深度学习的高能耗和可扩展性挑战,迫切需要将研究重点转向处理能耗约束。Tsetlin Machines(TM)是最近的一种机器学习方法,与神经网络等相比,它已经证明了显著降低的能量使用,同时在几个基准上表现出具有竞争力的准确性。然而,TM在很大程度上依赖于能量昂贵的随机数生成随机引导一组Tsetlin自动机到TM博弈的纳什均衡。在本文中,我们提出了一种新的有限状态学习自动机,可以取代Tsetlin自动机在TM学习,增加确定性。新的自动机使用多步确定性状态跳跃来加强子模式。同时,抛硬币跳过每$d$次状态更新确保了随机化的多样化。因此,$d$参数允许精细地控制随机化程度。例如,在一个示例中,$d=1$使每次更新都是随机的,而$d=\infty$使自动机完全确定。我们的实证结果表明,总体而言,只有相当程度的决定性降低准确性。能量方面,随机数生成构成TM的切换能量消耗,对于具有高$d$值的较大数据集节省高达11 mW的功率。因此,我们可以使用新的$d$ -参数来权衡精度和能耗,以促进低能耗的机器学习。
英文摘要:
Due to the high energy consumption and scalability challenges of deep learning, there is a critical need to shift research focus towards dealing with energy consumption constraints. Tsetlin Machines (TMs) are a recent approach to machine learning that has demonstrated significantly reduced energy usage compared to neural networks alike, while performing competitively accuracy-wise on several benchmarks. However, TMs rely heavily on energy-costly random number generation to stochastically guide a team of Tsetlin Automata to a Nash Equilibrium of the TM game. In this paper, we propose a novel finite-state learning automaton that can replace the Tsetlin Automata in TM learning, for increased determinism. The new automaton uses multi-step deterministic state jumps to reinforce sub-patterns. Simultaneously, flipping a coin to skip every $d$'th state update ensures diversification by randomization. The $d$-parameter thus allows the degree of randomization to be finely controlled. E.g., $d=1$ makes every update random and $d=\infty$ makes the automaton completely deterministic. Our empirical results show that, overall, only substantial degrees of determinism reduces accuracy. Energy-wise, random number generation constitutes switching energy consumption of the TM, saving up to 11 mW power for larger datasets with high $d$ values. We can thus use the new $d$-parameter to trade off accuracy against energy consumption, to facilitate low-energy machine learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。