基于方差-不变性-协方差正则化的神经主题模型自监督学习
Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization
AI总结：
提出一种结合方差-不变性-协方差正则化的自监督神经主题模型，通过正则化锚点与正样本表示及对抗性数据增强提升主题质量，在三个数据集上超越基线与最先进模型。
摘要：
在我们的研究中，我们提出了一种自监督神经主题模型（NTM），该模型结合了NTM和正则化自监督学习方法的优势，以提高性能。NTM利用神经网络学习文档中词语背后隐藏的潜在主题，与传统主题模型相比，具有更大的灵活性，并且能够估计出更连贯的主题。另一方面，一些自监督学习方法采用联合嵌入架构，使用两个相同的网络，为同一输入的两个增强版本生成相似的表示。对这些表示应用正则化以防止坍缩，否则网络会对所有输入输出恒定或冗余的表示。我们的模型通过显式正则化锚点样本和正样本的潜在主题表示来提升主题质量。我们还引入了一种对抗性数据增强方法，以替代启发式采样方法。我们进一步开发了几种变体模型，包括基于结合正负样本对比学习的NTM的模型。在三个数据集上的实验结果表明，我们的模型在定量和定性方面均优于基线和最先进的模型。
英文摘要：
In our study, we propose a self-supervised neural topic model (NTM) that combines the power of NTMs and regularized self-supervised learning methods to improve performance. NTMs use neural networks to learn latent topics hidden behind the words in documents, enabling greater flexibility and the ability to estimate more coherent topics compared to traditional topic models. On the other hand, some self-supervised learning methods use a joint embedding architecture with two identical networks that produce similar representations for two augmented versions of the same input. Regularizations are applied to these representations to prevent collapse, which would otherwise result in the networks outputting constant or redundant representations for all inputs. Our model enhances topic quality by explicitly regularizing latent topic representations of anchor and positive samples. We also introduced an adversarial data augmentation method to replace the heuristic sampling method. We further developed several variation models including those on the basis of an NTM that incorporates contrastive learning with both positive and negative samples. Experimental results on three datasets showed that our models outperformed baselines and state-of-the-art models both quantitatively and qualitatively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
