非对称压缩学习保证及其在量化草图中的应用
Asymmetric compressive learning guarantees with applications to quantized sketches
摘要:
压缩学习框架降低了在大规模数据集上训练的计算成本。在草图绘制阶段,数据首先被压缩到一个轻量级的草图向量,通过一个精心选择的特征图映射数据样本,并平均这些贡献。在学习阶段,然后通过解决优化问题从该草图中提取所需的模型参数,该优化问题还涉及特征图。当特征图在草图绘制和学习阶段相同时,正式的统计保证(超额风险界限)已经得到证明。然而,特征图的期望属性在草图绘制和学习期间是不同的(例如,分别是量化输出和可微性)。因此,我们研究的放松,这个地图是允许不同的每个阶段。首先,我们证明了现有的保证结转到这个不对称的计划,到一个受控的误差项,提供了一些有限的投影失真(LPD)的财产持有。然后,我们实例化这个框架的量化草图的设置,通过证明LPD确实持有二进制草图的贡献。最后,我们进一步验证的方法与数值模拟,包括大规模的应用程序中的音频事件分类。
英文摘要:
The compressive learning framework reduces the computational cost of training on large-scale datasets. In a sketching phase, the data is first compressed to a lightweight sketch vector, obtained by mapping the data samples through a well-chosen feature map, and averaging those contributions. In a learning phase, the desired model parameters are then extracted from this sketch by solving an optimization problem, which also involves a feature map. When the feature map is identical during the sketching and learning phases, formal statistical guarantees (excess risk bounds) have been proven. However, the desirable properties of the feature map are different during sketching and learning (e.g. quantized outputs, and differentiability, respectively). We thus study the relaxation where this map is allowed to be different for each phase. First, we prove that the existing guarantees carry over to this asymmetric scheme, up to a controlled error term, provided some Limited Projected Distortion (LPD) property holds. We then instantiate this framework to the setting of quantized sketches, by proving that the LPD indeed holds for binary sketch contributions. Finally, we further validate the approach with numerical simulations, including a large-scale application in audio event classification.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。