随机扰动最近邻搜索下对数线性模型的快速分期推理和学习
Fast Amortized Inference and Learning in Log-linear Models with Randomly Perturbed Nearest Neighbor Search
摘要：
在最坏的情况下，对数线性模型中的推断在输出空间的大小上线性缩放。当输出空间可行但非常大时，这通常是自然语言处理和计算机视觉任务中的瓶颈。本文提出了一种方法来推断具有次线性摊余成本的对数线性模型。我们的想法取决于使用Gumbel随机变量扰动和预先计算的最大内积搜索数据结构，以访问最可能的元素在次线性摊销时间。我们的方法产生可证明的运行时间和精度保证。此外，我们在ImageNet和词嵌入上进行了实证实验，显示出对数线性模型中的采样，推理和学习的显着加速。
英文摘要：
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amortized cost. Our idea hinges on using Gumbel random variable perturbations and a pre-computed Maximum Inner Product Search data structure to access the most-likely elements in sublinear amortized time. Our method yields provable runtime and accuracy guarantees. Further, we present empirical experiments on ImageNet and Word Embeddings showing significant speedups for sampling, inference, and learning in log-linear models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
