In this work, we generalize the information bottleneck (IB) approach to the multi-view learning context. The exponentially growing complexity of the optimal representation motivates the development of two novel formulations with more favorable performance-complexity tradeoffs. The first approach is based on forming a stochastic consensus and is suited for scenarios with significant {\em representation overlap} between the different views. The second method, relying on incremental updates, is tailored for the other extreme scenario with minimal representation overlap. In both cases, we extend our earlier work on the alternating directional methods of multiplier (ADMM) solver and establish its convergence and scalability. Empirically, we find that the proposed methods outperform state-of-the-art approaches in multi-view classification problems under a broad range of modelling parameters.
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
Institute for Communications Engineering, Technical University of Munich（慕尼黑工业大学通信工程研究所）；Signal Processing and Speech Communication Laboratory, Graz University of Technology（格拉茨技术大学信号处理与语音通信实验室）；Know-Center GmbH（Know-Center 有限公司）
OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment
Hong Kong University of Science and Technology（香港科学与技术大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；UiT – The Arctic University of Norway（挪威北极大学）；University of Copenhagen（哥本哈根大学）；Norwegian Computing Center（挪威计算中心）；University of Massachusetts Amherst（马萨诸塞大学阿姆赫斯特分校）
