简洁编码条件分布的兼容性问题的复杂性
On the Complexity of the Compatibility Problem for Succinctly Encoded Conditional Distributions
浏览论文内容
中文总结 AI 辅助
本文研究简洁编码为算术电路的条件分布的兼容性问题，证明其为co-NP完全或PSPACE完全问题，且存在兼容简洁条件分布的联合分布无法简洁表示，探讨了对概率建模和机器学习的意义。
中文摘要 AI 辅助
本文的研究动机是探究机器学习中概率模型所隐含的权衡关系。模型常以条件概率形式用于预测，但一对条件分布p(x|y)和p(y|x)可能与任何联合分布p(x,y)均不兼容。给定两个此类条件分布，判断是否存在兼容的联合分布的问题被称为兼容性问题。对于离散随机变量，当条件分布编码为概率表时，该兼容性问题存在已知的可计算处理的解决方案。本文将该问题的简洁版本形式化并展开研究，该版本将条件分布编码为算术电路，适用于高维场景下概率建模的实际应用，包括神经网络模型。研究表明，对于条件分布的简洁电路表示，兼容性问题是难解的：当所有概率均非零时，该问题为co-NP完全问题；当概率可以为零时，研究给出示例以区分多种兼容性概念，并证明该问题的多个版本为PSPACE完全问题。此外，研究还表明，假设多项式层次不崩溃，则存在兼容的简洁条件分布，其联合分布无法以简洁形式表示。本文讨论了这些结果对概率建模和机器学习的意义。
英文摘要
The motivation for this paper is the investigation of the trade-offs implicit in probabilistic models used in machine learning. Models are often used to make predictions in the form of conditional probabilities. However, a pair of conditional distributions p(x|y) and p(y|x) may not be compatible with any joint distribution p(x,y). Given two such conditionals, determining if there exists a compatible joint is known as the compatibility problem. For discrete random variables, when the conditionals are encoded as probability tables, the compatibility problem has a known solution, which is computationally tractable. In this paper, we formalise and study a succinct version of the problem, encoding conditional distributions as arithmetic circuits. This is applicable to practical applications of probabilistic modelling in high-dimensional settings, including neural network models. We show that, for succinct circuit representations of conditionals, the compatibility problem is intractable. In the case that all probabilities are non-zero, the problem is co-NP-complete. In the case that probabilities can be zero, we give examples to demonstrate that several notions of compatibility can be distinguished, and we prove that multiple versions of the problem are PSPACE-complete. Furthermore, we show that, assuming the polynomial hierarchy does not collapse, there exist compatible succinct conditionals whose joint cannot be expressed succinctly. Implications of these results for probabilistic modelling and machine learning are discussed.
