基于持续终身学习的神经主题建模
Neural Topic Modeling with Continual Lifelong Learning
AI总结：
针对短文档数据稀疏导致主题建模效果不佳的问题，提出终身神经主题建模（LNTM）框架，通过共享生成同源和选择性数据增强等方法持续积累主题并减少灾难性遗忘，在困惑度、主题连贯性和信息检索上取得显著提升。
摘要：
终身学习近期在构建机器学习系统方面引起了关注，这类系统能持续积累和转移知识以辅助未来的学习。无监督主题建模已被广泛用于从文档集合中发现主题。然而，由于数据稀疏性（例如在小型（短）文档集合中），主题建模的应用具有挑战性，从而生成不连贯的主题和次优的文档表示。为了解决这一问题，我们提出了一种用于神经主题建模的终身学习框架，该框架能够持续处理文档集合流，积累主题，并通过从多个来源进行知识转移来指导未来的主题建模任务，以更好地应对稀疏数据。在终身学习过程中，我们特别联合研究：（1）在生命周期内共享生成同源（潜在主题）以转移先验知识，以及（2）通过新颖的选择性数据增强、协同训练和主题正则化方法，最小化灾难性遗忘以保留过去的学习。给定一个文档集合流，我们将提出的终身神经主题建模（LNTM）框架应用于对三个稀疏文档集合进行建模作为未来任务，并展示了在困惑度、主题连贯性和信息检索任务上量化的性能提升。
英文摘要：
Lifelong learning has recently attracted attention in building machine learning systems that continually accumulate and transfer knowledge to help future learning. Unsupervised topic modeling has been popularly used to discover topics from document collections. However, the application of topic modeling is challenging due to data sparsity, e.g., in a small collection of (short) documents and thus, generate incoherent topics and sub-optimal document representations. To address the problem, we propose a lifelong learning framework for neural topic modeling that can continuously process streams of document collections, accumulate topics and guide future topic modeling tasks by knowledge transfer from several sources to better deal with the sparse data. In the lifelong process, we particularly investigate jointly: (1) sharing generative homologies (latent topics) over lifetime to transfer prior knowledge, and (2) minimizing catastrophic forgetting to retain the past learning via novel selective data augmentation, co-training and topic regularization approaches. Given a stream of document collections, we apply the proposed Lifelong Neural Topic Modeling (LNTM) framework in modeling three sparse document collections as future tasks and demonstrate improved performance quantified by perplexity, topic coherence and information retrieval task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
