通过离散扩散先验利用辅助信息进行主动数据采集
Active Data Acquisition with Side Information via Discrete Diffusion Priors
查看机构详情
- Oregon State University（俄勒冈州立大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
提出基于离散扩散先验的信息论主动采集框架，利用辅助信息优化掩码以最大化互信息，在MNIST、CIFAR-10和fastMRI上显著优于随机及现有方法。
中文摘要 AI 辅助
获取数据成本高昂：更高的测量保真度会消耗功率和存储空间，并可能收集无关内容，而激进地降低成本则可能丢弃后续分析所需的信息。我们通过一个信息论框架来解决这一权衡问题，该框架获取与广泛任务集相关的数据，而非针对单一模型。一个以辅助信息为条件的掩码策略，选择要测量的像素，以在预算下最大化离散图像与其部分观测之间的互信息；由于图像熵不依赖于掩码，这等价于最小化条件熵。一个冻结的离散去噪扩散模型（D3PM）提供后验，我们以两种方式使用它：作为训练一次性掩码生成器的熵替代指标，以及作为序列贪婪采集的标准。一次性生成器仅在谨慎处理下（包括用于二元掩码的无偏梯度估计器）才能优于随机掩码。在序列采集中，在MNIST上，先验在10%预算下比随机产生的错误少8倍；在CIFAR-10上，它获得了0.9–3.4 dB的增益。在fastMRI上，我们提出的使用静态掩码的技术优于诸如可变密度和LOUPE等知名方法。
英文摘要
Acquiring data is costly: higher measurement fidelity costs power and storage and risks collecting irrelevant content, while aggressive cost reduction can discard information that later analysis needs. We address this trade-off with an information-theoretic framework that acquires data relevant to a broad set of tasks rather than to one model. A mask policy, conditioned on side information, chooses which pixels to measure so as to maximize the mutual information between a discrete image and its partial observation under a budget; since the image entropy does not depend on the mask, this is equivalent to minimizing the conditional entropy. A frozen discrete denoising diffusion model (D3PM) supplies the posterior, and we use it in two ways: as an entropy surrogate for training a one-shot mask generator, and as the criterion for sequential greedy acquisition. The one-shot generator outperforms random masks only with care, including an unbiased gradient estimator for binary masks. With sequential acquisition, on MNIST the prior makes $8\times$ fewer errors than random at a $10\%$ budget, and on CIFAR-10 it gains $0.9$--$3.4$~dB. On fastMRI, our proposed technique using a static mask outperforms the well-known methods such as variable density and LOUPE.
