In this paper we present a foundational study on a constrained method that defines learning problems with Neural Networks in the context of the principle of least cognitive action, which very much resembles the principle of least action in mechanics. Starting from a general approach to enforce constraints into the dynamical laws of learning, this work focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. In particular, the structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data, leading to "architectural" and "input-related" constraints, respectively. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner, that makes this study an important step toward alternative ways of processing continuous streams of data with Neural Networks. The connection with the classic Backpropagation-based update rule of the weights of networks is discussed, showing that there are conditions under which our approach degenerates to Backpropagation. Moreover, the theory is experimentally evaluated on a simple problem that allows us to deeply study several aspects of the theory itself and to show the soundness of the model.
Developing Lagrangian-based Methods for Nonsmooth Nonconvex Optimization
Institute of Operational Research and Analytics, National University of Singapore（新加坡国立大学运营研究与分析研究所）；School of Computer and Computing Science, Hangzhou City University（杭州城市大学计算机与计算科学学院）
Unit-Aware Genetic Programming for the Development of Empirical Equations
Faculty of Computer Science, Otto-von-Guericke-Universität Magdeburg（马格德堡奥托·冯·格里克大学计算机科学学院）；Interdisciplinary Center for Scientific Computing, Universität Heidelberg（海德堡大学跨学科科学计算中心）
Mapping Informal Settlements in Developing Countries with Multi-resolution, Multi-spectral Data
DFKI（德国人工智能研究中心）；TU Kaiserlautern（凯泽斯劳滕工业大学）；University of Oxford（牛津大学）；DLR（德国航空航天中心）；King’s College London（伦敦国王学院）；Czech Geological Survey（捷克地质调查局）；University of Warsaw（华沙大学）
