The search for universal laws that help establish a relationship between dynamics and computation is driven by recent expansionist initiatives in biologically inspired computing. A general setting to understand both such dynamics and computation is a driven dynamical system that responds to a temporal input. Surprisingly, we find memory-loss a feature of driven systems to forget their internal states helps provide unambiguous answers to the following fundamental stability questions that have been unanswered for decades: what is necessary and sufficient so that slightly different inputs still lead to mostly similar responses? How does changing the driven system's parameters affect stability? What is the mathematical definition of the edge-of-criticality? We anticipate our results to be timely in understanding and designing biologically inspired computers that are entering an era of dedicated hardware implementations for neuromorphic computing and state-of-the-art reservoir computing applications.
Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks
University of Bologna（博洛尼亚大学）；Politecnico di Torino（都灵理工大学）
神经架构搜索（NAS）越来越受欢迎，以自动探索深度学习（DL）架构的准确性与计算复杂性的权衡。当针对微小的边缘设备时，DL部署的主要挑战是匹配严格的内存约束，因此大多数NAS算法将模型大小视为复杂性度量。其他方法通过权衡推理操作的准确性和数量来减少DL模型的能量或延迟。能量和内存很少被同时考虑，特别是低搜索成本的差分NAS（DNAS）解决方案。我们克服了这一限制，提出了第一个DNAS，直接解决最现实的情况下，从设计师的角度来看：共同优化的准确性和能源（或延迟）的内存约束下，由目标硬件。我们通过在训练期间将两个复杂度相关的损失函数与独立强度相结合来做到这一点。通过对MLPerf Tiny基准测试套件中的三个边缘相关任务进行测试，我们在能量与精度空间中获得了丰富的Pareto架构集，内存占用限制范围为基线网络的75%至6.25%。当部署在商业边缘设备STM NUCLEO-H743 ZI 2上时，我们的网络在相同的内存限制下，能耗范围为2.18倍，准确度为4.04%，并将能耗降低高达2.2倍，相对于基线的准确度下降可以忽略不计。
