Network representation learning (NRL) technique has been successfully adopted in various data mining and machine learning applications. Random walk based NRL is one popular paradigm, which uses a set of random walks to capture the network structural information, and then employs word2vec models to learn the low-dimensional representations. However, until now there is lack of a framework, which unifies existing random walk based NRL models and supports to efficiently learn from large networks. The main obstacle comes from the diverse random walk models and the inefficient sampling method for the random walk generation. In this paper, we first introduce a new and efficient edge sampler based on Metropolis-Hastings sampling technique, and theoretically show the convergence property of the edge sampler to arbitrary discrete probability distributions. Then we propose a random walk model abstraction, in which users can easily define different transition probability by specifying dynamic edge weights and random walk states. The abstraction is efficiently supported by our edge sampler, since our sampler can draw samples from unnormalized probability distribution in constant time complexity. Finally, with the new edge sampler and random walk model abstraction, we carefully implement a scalable NRL framework called UniNet. We conduct comprehensive experiments with five random walk based NRL models over eleven real-world datasets, and the results clearly demonstrate the efficiency of UniNet over billion-edge networks.
National Research University Higher School of Economics（国家研究型高等经济学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Samsung AI Center Moscow（三星莫斯科人工智能中心）；Samsung-HSE Laboratory（三星-高等经济学院实验室）；HSE（高等经济学院）；Skoltech（斯科尔科沃科学技术研究院）
Metropolis-Hastings view on variational inference and adversarial training
National Research University Higher School of Economics（国家研究型高等经济学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Samsung AI Center Moscow（三星莫斯科人工智能中心）；HSE（高等经济学院）；Skoltech（斯科尔科沃科学技术研究院）
Differentiating Metropolis-Hastings to Optimize Intractable Densities
Massachusetts Institute of Technology（麻省理工学院）；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；University of Oxford（牛津大学）；JuliaHub；Pumas-AI
Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings
Carnegie Mellon University（卡内基梅隆大学）；Deepmind（深度思维）；UC San Diego（加州大学圣地亚哥分校）
虽然最近的工作表明，由无处不在的掩蔽语言建模（MLM）目标训练的模型的分数有效地区分了可能的序列和不可能的序列，但这些MLM是否指定了可能序列空间上的原则概率分布仍然是一个悬而未决的问题。在本文中，我们解释MLMs的能量为基础的序列模型，并提出了两个能源参数化可从训练MLMs。为了正确地从这些模型中抽取样本，我们开发了一个基于Metropolis-Hastings Monte Carlo算法的易处理的抽样方案。在我们的方法中，从用于训练掩蔽语言模型的相同掩蔽条件中提出样本，并且根据目标分布基于它们的能量值来接受或拒绝它们。我们通过探索从这些基于能量的模型中提取的样本的质量来验证所提出的参数化的有效性，这些样本用于开放式无条件生成和机器翻译的条件生成任务。我们从理论上和经验上证明了我们的采样算法，表明掩蔽条件本身不会产生其平稳分布是我们的目标分布的马尔可夫链，并且我们的方法比最近提出的其他无向生成方法生成更高质量的样本（Wang et al.，2019年，Ghazvininejad等人，2019年）。
QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation
Instituto de Telecomunicações（电信研究所）；University of Washington（华盛顿大学）；Instituto Superior Técnico（高等技术学院）；Universidade de Lisboa（里斯本大学）；Unbabel（Unbabel公司）；ELLIS Unit Lisbon（里斯本ELLIS单元）
