测度到测度算子的推前映射通用逼近
Universal Approximation of Measure-to-Measure Operators by Pushforwards
- Doshisha University（同志社大学）
- RIKEN AIP（理化学研究所人工智能研究中心）
- UT Austin（得克萨斯大学奥斯汀分校）
- UCLA（加利福尼亚大学洛杉矶分校）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文研究测度到测度算子的推前映射通用逼近，引入均匀水平集条件，证明在满足条件的紧族上可均匀逼近，并推广至变换器与交叉注意力架构。
AI中文摘要：
许多学习任务将输入分布映射到输出分布。对此类算子进行建模的一种自然方式是使用可能依赖于整个输入分布的连续函数变换每个输入样本，然后取变换后样本的分布。这定义了一个依赖于测度的推前模型，并包含了变换器的测度论表述。我们研究此类模型何时能够逼近概率测度空间之间的任意连续算子。我们首先证明，当允许原子输入时，通用逼近会失败：某些连续测度到测度算子会分裂或重新分配原子质量，无法被确定性推前模型任意好地逼近。然后，我们引入均匀水平集条件，该条件要求存在一个连续的依赖于测度的标量化函数，其收缩的水平集邻域在输入族上携带均匀消失的质量。特别是，绝对连续测度的紧族满足此条件。在满足此条件的每个紧族上，我们证明任何输出具有有限p阶矩的连续测度到测度算子都可以在p-Wasserstein距离下被连续的依赖于测度的推前映射均匀逼近。将我们的定理与现有的依赖于测度的上下文内映射的逼近结果相结合，可以得到测度论变换器的通用逼近。我们还将框架扩展到连续变化的源测度，从而为与交叉注意力架构紧密对齐的一类推前模型获得相应的通用性结果。
英文摘要：
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite $p$-th moment can be uniformly approximated, in the $p$-Wasserstein distance, by continuous measure-dependent pushforwards. Combining our theorem with existing approximation results for measure-dependent in-context maps yields universal approximation by measure-theoretic transformers. We also extend the framework to continuously-varying source measures, yielding a corresponding universality result for a class of pushforward models that are closely aligned with cross-attention architectures.
