The feasibility of variational quantum algorithms, the most popular correspondent of neural networks on noisy, near-term quantum hardware, is highly impacted by the circuit depth of the involved parametrized quantum circuits (PQCs). Higher depth increases expressivity, but also results in a detrimental accumulation of errors. Furthermore, the number of parameters involved in the PQC significantly influences the performance through the necessary number of measurements to evaluate gradients, which scales linearly with the number of parameters.
Motivated by this, we look at deep equilibrium models (DEQs), which mimic an infinite-depth, weight-tied network using a fraction of the memory by employing a root solver to find the fixed points of the network. In this work, we present Quantum Deep Equilibrium Models (QDEQs): a training paradigm that learns parameters of a quantum machine learning model given by a PQC using DEQs. To our knowledge, no work has yet explored the application of DEQs to QML models. We apply QDEQs to find the parameters of a quantum circuit in two settings: the first involves classifying MNIST-4 digits with 4 qubits; the second extends it to 10 classes of MNIST, FashionMNIST and CIFAR. We find that QDEQ is not only competitive with comparable existing baseline models, but also achieves higher performance than a network with 5 times more layers. This demonstrates that the QDEQ paradigm can be used to develop significantly more shallow quantum circuits for a given task, something which is essential for the utility of near-term quantum computers.
Our code is available at https://github.com/martaskrt/qdeq.
Generating equilibrium molecules with deep neural networks
Technische Universität Berlin（柏林工业大学）
发现具有理想性质的原子体系是化学和材料科学中的一个重大挑战。在这里，我们介绍了一种新的自回归卷积深度神经网络架构，它通过在三维空间中顺序放置原子来生成分子平衡结构。该模型用易于处理的条件概率估计了分子构型上的联合概率，条件概率仅取决于原子与其核电荷之间的距离。它将最先进的原子神经网络的概念与图像和语音的自回归生成模型相结合。我们证明，该架构是能够产生接近平衡的C $_7$ O $_2$ H $_{10}$的结构异构体的分子。
Peking University（北京大学）；Pazhou Lab（琶洲实验室）；Swiss Federal Institute of Technology（瑞士联邦理工学院）；Nanjing University of Information Science & Technology（南京信息工程大学）
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息学院）；School of Electronic Information（电子信息学院）；Communications, Huazhong University of Science（华中科技大学通信学院）；School of Science, Wuhan University of Technology（武汉理工大学理学院）
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Unité Mixte de Physique（联合物理实验室）；Thales（泰雷兹集团）；Mila（米拉研究所）；Université de Montréal（蒙特利尔大学）；Canadian Institute for Advanced Research（加拿大高等研究院）；Google（谷歌）
Large-Scale Contextual Market Equilibrium Computation through Deep Learning
PKU-WUHAN Institute for Artificial Intelligence（北京大学武汉研究院）；CFCS, School of Computer Science, Peking University（计算机科学系，北京大学）；Yuanpei College, Peking University（北京大学元培学院）；School of Computer Science, Wuhan University（武汉大学计算机科学系）；CMAR, Institute for AI, Peking University（人工智能研究院，北京大学）
A Closer Look at the Adversarial Robustness of Deep Equilibrium Models
Tsinghua University（清华大学）；Sea AI Lab（Sea人工智能实验室）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
Online Deep Equilibrium Learning for Regularization by Denoising
Washington University in St. Louis（圣路易斯华盛顿大学）
即插即用先验（Plug-and-Play Priors，PPENS）和去噪正则化（Regularization by Denoising，RED）是通过计算结合物理测量模型和学习的图像先验的算子的不动点来解决成像逆问题的广泛使用的框架。虽然传统的PnP/RED公式集中在使用图像降噪器指定的先验知识上，但人们对学习端到端最佳PnP/RED先验知识的兴趣越来越大。最近的深度均衡模型（DEQ）框架通过隐式差分定点方程，实现了PPENDIX/RED先验的存储高效的端到端学习，而无需存储中间激活值。然而，依赖的计算/存储器的复杂性的测量模型在PPENS/RED的测量总数离开DEQ不切实际的许多成像应用。我们提出ODER作为一种新的策略，通过随机近似的测量模型，提高效率的DEQ。我们从理论上分析了ODER，深入了解其收敛性和逼近传统DEQ方法的能力。我们的数值结果表明，由于ODER在三个不同的成像应用中的训练/测试复杂性的潜在改进。
Gradient descent dynamics for deep equilibrium models
Carnegie Mellon University, Department of Statistics（卡内基梅隆大学统计系）；Carnegie Mellon University, Department of Statistics and Machine Learning（卡内基梅隆大学统计与机器学习系）
