发表机构
imec（imec）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出新方法优化深度可微逻辑门网络和查找表网络连接，利用概率分布选最优连接，并行学习门类型等。在多个基准测试中表现优于固定连接网络，减少门数量，确保多层训练稳定，还提出LUT神经元描述，提升模型性能。
AI 中文摘要
我们介绍了一种用于深度可微逻辑门网络（LGNs）和查找表网络（LUTNs）连接的部分和完全优化的新方法。训练方法利用每个门/查找表（LUT）输入引脚的一组连接上的概率分布，选择优点最高的连接，同时并行学习最优门类型或LUT条目。在Yin-Yang、MNIST手写数字和Fashion-MNIST基准测试中，连接优化的LGNs优于标准固定连接LGNs，且所需逻辑门数量更少。通过采用高学习率、直通估计器和修剪恒定输出门类型，确保了高达十层的训练稳定性。还提出了一种LUT神经元描述，实现了通过反向传播的稳定训练。与固定连接LGN训练算法相比，该模型所需可训练参数减少四倍且准确率更高。连接训练算法对LUTNs也有效，两层2000个6输入LUT的准确率达98.88%。
英文摘要
We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.
