基于受MIMO启发的DPP MAP推断在带宽受限多源数据上的学习
Learning on Bandwidth Constrained Multi-Source Data with MIMO-inspired DPP MAP Inference
AI总结:
本文提出一种基于MIMO通信技术的分布式DPP MAP推断方法,在带宽受限下通过保行列式稀疏表示与轻量级反馈信道实现多源数据多样性选择,无需交换原始数据,并在多个数据集上显著优于基线方法。
摘要:
本文提出了行列式点过程(DPP)推断的分布式版本,以增强有限通信带宽下的多源数据多样化。DPP是一种流行的概率方法,通过强制所选子集中元素的排斥性来提高数据多样性。DPP中被广泛研究的最大后验(MAP)推断旨在识别由DPP量化的具有最高多样性的子集。然而,这种方法受限于所有数据样本都在一个点可用的假设,这阻碍了其在现实世界应用中的适用性,例如交通数据集中数据样本分布在多个源且它们之间的通信是带宽受限的。受多输入多输出(MIMO)通信系统中使用的技术启发,我们提出了一种在分布式源之间执行MAP推断的策略。具体而言,我们证明了多样性最大化的分布式样本选择问题的下界可以被视为MIMO系统中的功率分配问题。我们使用所选样本的保行列式稀疏表示来执行本地源的样本预编码,以便由DPP处理。我们的方法不需要源之间交换原始数据,而是需要一个带宽受限的反馈信道,从中心向数据源发送轻量级的多样性度量,类似于MIMO系统中的CSI消息。实验表明,我们这种可扩展的方法在i.i.d和非i.i.d设置下均能优于基线方法,包括随机选择、无反馈的未知情个体DPP以及基于SVD反馈的DPP。具体而言,它在CIFAR-10、CIFAR-100、StanfordCars和GTSRB数据集的潜在表示中实现了1到6个对数差异的多样性增益。
英文摘要:
This paper proposes a distributed version of Determinant Point Processing (DPP) inference to enhance multi-source data diversification under limited communication bandwidth. DPP is a popular probabilistic approach that improves data diversity by enforcing the repulsion of elements in the selected subsets. The well-studied Maximum A Posteriori (MAP) inference in DPP aims to identify the subset with the highest diversity quantified by DPP. However, this approach is limited by the presumption that all data samples are available at one point, which hinders its applicability to real-world applications such as traffic datasets where data samples are distributed across sources and communication between them is band-limited. Inspired by the techniques used in Multiple-Input Multiple-Output (MIMO) communication systems, we propose a strategy for performing MAP inference among distributed sources. Specifically, we show that a lower bound of the diversity-maximized distributed sample selection problem can be treated as a power allocation problem in MIMO systems. A determinant-preserved sparse representation of selected samples is used to perform sample precoding in local sources to be processed by DPP. Our method does not require raw data exchange among sources, but rather a band-limited feedback channel to send lightweight diversity measures, analogous to the CSI message in MIMO systems, from the center to data sources. The experiments show that our scalable approach can outperform baseline methods, including random selection, uninformed individual DPP with no feedback, and DPP with SVD-based feedback, in both i.i.d and non-i.i.d setups. Specifically, it achieves 1 to 6 log-difference diversity gain in the latent representation of CIFAR-10, CIFAR-100, StanfordCars, and GTSRB datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。