SORT: Second-Order Response Transform for Visual Recognition
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University(上海交通大学协同媒体创新中心);The Johns Hopkins University(约翰斯·霍普金斯大学);The University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校)
Yan Wang, Lingxi Xie, Chenxi Liu, Ya Zhang, Wenjun Zhang, Alan Yuille
In this paper, we reveal the importance and benefits of introducing second-order operations into deep neural networks. We propose a novel approach named Second-Order Response Transform (SORT), which appends element-wise product transform to the linear sum of a two-branch network module. A direct advantage of SORT is to facilitate cross-branch response propagation, so that each branch can update its weights based on the current status of the other branch. Moreover, SORT augments the family of transform operations and increases the nonlinearity of the network, making it possible to learn flexible functions to fit the complicated distribution of feature space. SORT can be applied to a wide range of network architectures, including a branched variant of a chain-styled network and a residual network, with very light-weighted modifications. We observe consistent accuracy gain on both small (CIFAR10, CIFAR100 and SVHN) and big (ILSVRC2012) datasets. In addition, SORT is very efficient, as the extra computation overhead is less than 5%.
Automatic Recognition of Coal and Gangue based on Convolution Neural Network
Institute of Huaqiao University(华侨大学研究院);Industrial Engineering Research Center, universities and intelligent systems in Fujian Province(福建省高校智能系统工业工程研究中心)