为什么网络具有抑制性/负连接？
Why do networks have inhibitory/negative connections?
AI总结：
本文从表示能力视角解释大脑抑制性连接与深度网络负权重的必要性，证明非负权重网络在非递减激活下不具备通用逼近性，并揭示其无法表示的几何性质。
摘要：
为什么大脑具有抑制性连接？为什么深度网络具有负权重？我们从表示能力的角度提出一个答案。我们认为，表示函数既是（i）自然智能中大脑的主要作用，也是（ii）人工智能中深度网络的主要作用。对于为什么存在抑制性/负权重，我们的答案是：为了学习更多函数。我们证明，在没有负权重的情况下，采用非递减激活函数的神经网络不是通用逼近器。尽管这对某些人而言可能是一个直观的结果，但据我们所知，无论在机器学习还是神经科学中，都还没有形式化理论说明负权重为何在表示能力的语境下至关重要。此外，我们还对非负深度网络无法表示的表示空间的几何性质给出了洞见。我们期望这些洞见将促使人们更深入地理解施加于权重分布上的更复杂归纳先验，这些先验可带来更高效的生物学习和机器学习。
英文摘要：
Why do brains have inhibitory connections? Why do deep networks have negative weights? We propose an answer from the perspective of representation capacity. We believe representing functions is the primary role of both (i) the brain in natural intelligence, and (ii) deep networks in artificial intelligence. Our answer to why there are inhibitory/negative weights is: to learn more functions. We prove that, in the absence of negative weights, neural networks with non-decreasing activation functions are not universal approximators. While this may be an intuitive result to some, to the best of our knowledge, there is no formal theory, in either machine learning or neuroscience, that demonstrates why negative weights are crucial in the context of representation capacity. Further, we provide insights on the geometric properties of the representation space that non-negative deep networks cannot represent. We expect these insights will yield a deeper understanding of more sophisticated inductive priors imposed on the distribution of weights that lead to more efficient biological and machine learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
