通过软去相关实现的可扩展且有效的深度CCA
Scalable and Effective Deep CCA via Soft Decorrelation
摘要：
最近，广泛使用的多视图学习模型典型相关分析（CCA）已通过深度神经网络推广到非线性环境。现有的深度CCA模型通常在不同视图在公共潜在空间中最大程度地相关之前首先对每个视图的特征维度进行去相关。这种特征去相关是通过强制执行精确的去相关约束来实现的;由于在每次训练迭代中精确去相关所需的矩阵求逆或SVD运算，因此这些模型在计算上是昂贵的。此外，去相关步骤通常与基于梯度下降的优化分离，导致次优解。我们提出了一种新的深度CCA模型软CCA来克服这些问题。具体地说，精确解相关被软解相关取代，通过基于小批量的随机解相关损失（SDL）与其他训练目标联合优化。大量的实验表明，所提出的软CCA比现有的深度CCA模型更有效和高效。此外，我们的SDL损失可以应用于多视图学习之外的其他深度模型，并且与现有的去相关损失相比具有更好的性能。
英文摘要：
Recently the widely used multi-view learning model, Canonical Correlation Analysis (CCA) has been generalised to the non-linear setting via deep neural networks. Existing deep CCA models typically first decorrelate the feature dimensions of each view before the different views are maximally correlated in a common latent space. This feature decorrelation is achieved by enforcing an exact decorrelation constraint; these models are thus computationally expensive due to the matrix inversion or SVD operations required for exact decorrelation at each training iteration. Furthermore, the decorrelation step is often separated from the gradient descent based optimisation, resulting in sub-optimal solutions. We propose a novel deep CCA model Soft CCA to overcome these problems. Specifically, exact decorrelation is replaced by soft decorrelation via a mini-batch based Stochastic Decorrelation Loss (SDL) to be optimised jointly with the other training objectives. Extensive experiments show that the proposed soft CCA is more effective and efficient than existing deep CCA models. In addition, our SDL loss can be applied to other deep models beyond multi-view learning, and obtains superior performance compared to existing decorrelation losses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
