arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.13426cs.LGq-bio.NCstat.ML
收藏

离散隐变量模型的贝叶斯主动学习

Bayesian Active Learning for Discrete Latent Variable Models

Princeton University(普林斯顿大学);Princeton Neuroscience Institute(普林斯顿神经科学研究所)

Aditi Jha, Zoe C. Ashwood, Jonathan W. Pillow

AI总结:

针对主动学习在隐变量模型中被忽视的问题,提出离散隐变量回归模型的最大互信息输入选择框架,应用于MLR和GLM-HMM,显著减少拟合数据量并优于近似方法。

摘要:

主动学习旨在减少拟合模型参数所需的数据量,因此构成了现代机器学习中一类重要的技术。然而,过去关于主动学习的研究在很大程度上忽略了隐变量模型,而该模型在神经科学、心理学以及各种其他工程和科学学科中发挥着至关重要的作用。在这里,我们通过提出一种用于离散隐变量回归模型的最大互信息输入选择的新框架来解决这一空白。我们首先将我们的方法应用于一类被称为“线性回归混合”(MLR)的模型。虽然众所周知主动学习对于线性高斯回归模型没有优势,但我们使用Fisher信息解析地表明,主动学习对于此类模型的混合仍然可以实现巨大的收益,并且我们通过模拟和真实世界数据验证了这种改进。然后,我们考虑一类强大的时间结构化隐变量模型,由带有广义线性模型(GLM)观测的隐马尔可夫模型(HMM)给出,该模型最近被用于从动物决策数据中识别离散状态。我们表明,我们的方法大幅减少了拟合GLM-HMM所需的数据量,并且优于基于变分和摊销推理的各种近似方法。因此,针对隐变量模型的信息最大化学习为表征时间结构化的隐状态提供了一种强大的工具,在神经科学及其他领域具有广泛的应用。

英文摘要:

Active learning seeks to reduce the amount of data required to fit the parameters of a model, thus forming an important class of techniques in modern machine learning. However, past work on active learning has largely overlooked latent variable models, which play a vital role in neuroscience, psychology, and a variety of other engineering and scientific disciplines. Here we address this gap by proposing a novel framework for maximum-mutual-information input selection for discrete latent variable regression models. We first apply our method to a class of models known as "mixtures of linear regressions" (MLR). While it is well known that active learning confers no advantage for linear-Gaussian regression models, we use Fisher information to show analytically that active learning can nevertheless achieve large gains for mixtures of such models, and we validate this improvement using both simulations and real-world data. We then consider a powerful class of temporally structured latent variable models given by a Hidden Markov Model (HMM) with generalized linear model (GLM) observations, which has recently been used to identify discrete states from animal decision-making data. We show that our method substantially reduces the amount of data needed to fit GLM-HMM, and outperforms a variety of approximate methods based on variational and amortized inference. Infomax learning for latent variable models thus offers a powerful for characterizing temporally structured latent states, with a wide variety of applications in neuroscience and beyond.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑