arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2002.10435cs.LGcs.DSstat.ML
收藏

从不受信任的批次中学习结构化分布:更快,更简单

Learning Structured Distributions From Untrusted Batches: Faster and Simpler

Massachusetts Institute of Technology(麻省理工学院);Microsoft Research AI(微软研究院人工智能部门)

Sitan Chen, Jerry Li, Ankur Moitra

摘要:

我们重新讨论了由Qiao和Valiant [QV 17]引入的从不可信批次中学习的问题。最近,Jain和Orlitsky [JO 19]给出了一个简单的半定规划方法,该方法基于割范数,在多项式时间内实现了基本上信息理论上的最优误差。同时,Chen等人[CLM 19]考虑了问题的一个变体,其中假设$μ$是结构化的,例如对数凹、单调风险率、$t$ -模态等。在这种情况下,可以在$n$中实现样本复杂度次线性的相同误差,并且他们使用Haar小波展示了一种准多项式时间算法。在本文中,我们找到了一种有吸引力的方法来综合[JO 19]和[CLM 19]的技术,以提供两全其美的效果:一种在多项式时间内运行的算法,可以利用底层分布中的结构来实现次线性样本复杂度。一路上,我们简化了[JO 19]的方法,避免了SDP舍入的需要,并通过软滤波的镜头对其进行了更直接的解释,软滤波是一种强大的高维鲁棒估计技术。我们验证了我们的算法在初步的实验评估的有用性。

英文摘要:

We revisit the problem of learning from untrusted batches introduced by Qiao and Valiant [QV17]. Recently, Jain and Orlitsky [JO19] gave a simple semidefinite programming approach based on the cut-norm that achieves essentially information-theoretically optimal error in polynomial time. Concurrently, Chen et al. [CLM19] considered a variant of the problem where $μ$ is assumed to be structured, e.g. log-concave, monotone hazard rate, $t$-modal, etc. In this case, it is possible to achieve the same error with sample complexity sublinear in $n$, and they exhibited a quasi-polynomial time algorithm for doing so using Haar wavelets. In this paper, we find an appealing way to synthesize the techniques of [JO19] and [CLM19] to give the best of both worlds: an algorithm which runs in polynomial time and can exploit structure in the underlying distribution to achieve sublinear sample complexity. Along the way, we simplify the approach of [JO19] by avoiding the need for SDP rounding and giving a more direct interpretation of it through the lens of soft filtering, a powerful recent technique in high-dimensional robust estimation. We validate the usefulness of our algorithms in preliminary experimental evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑