arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1703.06993cs.CV
收藏

SORT:用于视觉识别的二阶响应变换

SORT: Second-Order Response Transform for Visual Recognition

Cooperative Medianet Innovation Center, Shanghai Jiao Tong University(上海交通大学协同媒体创新中心);The Johns Hopkins University(约翰斯·霍普金斯大学);The University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校)

Yan Wang, Lingxi Xie, Chenxi Liu, Ya Zhang, Wenjun Zhang, Alan Yuille

摘要:

在本文中,我们揭示了将二阶运算引入深度神经网络的重要性和好处。本文提出了一种新的二阶响应变换方法(SORT),该方法将逐元素乘积变换附加到两个分支网络模块的线性和上。SORT的一个直接优点是便于跨分支响应传播,这样每个分支都可以根据其他分支的当前状态更新其权重。此外,SORT扩充了变换运算族,增加了网络的非线性,使得学习灵活的函数以适应特征空间的复杂分布成为可能。SORT可以应用于广泛的网络架构,包括链式网络和剩余网络的分支变体,具有非常轻的修改。我们在小型(CIFAR10,CIFAR100和SVHN)和大型(ILSVRC 2012)数据集上观察到一致的准确性增益。此外,SORT是非常有效的,因为额外的计算开销小于5%。

英文摘要:

In this paper, we reveal the importance and benefits of introducing second-order operations into deep neural networks. We propose a novel approach named Second-Order Response Transform (SORT), which appends element-wise product transform to the linear sum of a two-branch network module. A direct advantage of SORT is to facilitate cross-branch response propagation, so that each branch can update its weights based on the current status of the other branch. Moreover, SORT augments the family of transform operations and increases the nonlinearity of the network, making it possible to learn flexible functions to fit the complicated distribution of feature space. SORT can be applied to a wide range of network architectures, including a branched variant of a chain-styled network and a residual network, with very light-weighted modifications. We observe consistent accuracy gain on both small (CIFAR10, CIFAR100 and SVHN) and big (ILSVRC2012) datasets. In addition, SORT is very efficient, as the extra computation overhead is less than 5%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑