本文通过Monte Carlo Arithmetic与VPREC评估蛋白质功能预测模型DeepGOPlus推理阶段的浮点扰动稳定性及降精度可行性,证明其数值可靠但低精度部署需选择性进行。
摘要:
卷积神经网络(CNN)目前是应用最广泛的深度神经网络(DNN)架构之一,并在许多问题上取得了最先进的性能。CNN最初应用于计算机视觉任务,但除图像外,它也能很好地处理任何具有空间关系的数据,并已被应用于不同领域。然而,近期研究强调了DNN中的数值稳定性挑战,这也与其已知的对噪声注入的敏感性有关。这些挑战可能危及DNN的性能和可靠性。本文研究了用于预测蛋白质功能的CNN——DeepGOPlus。DeepGOPlus已取得最先进的性能,并能够成功利用和注释蛋白质组学中大量涌现的蛋白质序列。我们通过量化由底层浮点数据扰动产生的数值不确定性,来确定该模型推理阶段的数值稳定性。此外,我们探索了在DeepGOPlus推理中使用降精度浮点格式以减少内存消耗和延迟的机会。这是通过使用Monte Carlo Arithmetic(一种通过实验量化浮点运算误差的技术)和VPREC(一种以可自定义浮点精度格式模拟结果的工具)对DeepGOPlus的执行进行插桩来实现的。研究重点放在推理阶段,因为它是DeepGOPlus模型的主要交付物,可在不同环境中广泛应用。总体而言,我们的结果表明,尽管DeepGOPlus CNN在数值上非常稳定,但它只能选择性地以较低精度浮点格式实现。我们得出结论:从预训练DeepGOPlus模型获得的预测在数值上非常可靠,并且高效地使用了现有浮点格式。
英文摘要:
Convolutional neural networks (CNNs) are currently among the most widely-used deep neural network (DNN) architectures available and achieve state-of-the-art performance for many problems. Originally applied to computer vision tasks, CNNs work well with any data with a spatial relationship, besides images, and have been applied to different fields. However, recent works have highlighted numerical stability challenges in DNNs, which also relates to their known sensitivity to noise injection. These challenges can jeopardise their performance and reliability. This paper investigates DeepGOPlus, a CNN that predicts protein function. DeepGOPlus has achieved state-of-the-art performance and can successfully take advantage and annotate the abounding protein sequences emerging in proteomics. We determine the numerical stability of the model's inference stage by quantifying the numerical uncertainty resulting from perturbations of the underlying floating-point data. In addition, we explore the opportunity to use reduced-precision floating point formats for DeepGOPlus inference, to reduce memory consumption and latency. This is achieved by instrumenting DeepGOPlus' execution using Monte Carlo Arithmetic, a technique that experimentally quantifies floating point operation errors and VPREC, a tool that emulates results with customizable floating point precision formats. Focus is placed on the inference stage as it is the primary deliverable of the DeepGOPlus model, widely applicable across different environments. All in all, our results show that although the DeepGOPlus CNN is very stable numerically, it can only be selectively implemented with lower-precision floating-point formats. We conclude that predictions obtained from the pre-trained DeepGOPlus model are very reliable numerically, and use existing floating-point formats efficiently.
HDTree: Generative Modeling of Cellular Hierarchies for Robust Lineage Inference
Centre for Artificial Intelligence and Robotics(人工智能与机器人中心);Hong Kong Institute of Science and Innovation(香港创新科学研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Engineering, Westlake University(西湖大学工程学院)
Stability of Linear Structural Equation Models of Causal Inference
University of Maryland, College Park(马里兰大学帕克分校);Indian Institute of Science, Bangalore(印度科学理工学院班加罗尔分校);Microsoft Research, India(微软印度研究院)
研究了因果推理的线性结构方程模型($\LSEM$)中参数恢复问题的数值稳定性。从Wright(1920)开始的一长串工作集中于理解$\LSEM$的哪些子类允许有效的参数恢复。尽管经过了几十年的研究,这个问题仍未完全解决。本文的目标是补充这条线的工作,我们想了解的情况下,有效的恢复是可能的恢复问题的稳定性。Pearl因果关系概念的数值稳定性首先在Schulman和Srivastava(2016)中使用条件数的概念进行了研究,其中他们提供了病态的例子。在这项工作中,我们为$\LSEM$提供了条件数分析。首先证明了在一个充分条件下,对于$\LSEM$的某个子类\emph{bow-free}(Brito and Pearl(2002)),参数恢复是稳定的.我们进一步证明,\emph{randomly}选择的输入参数为这个家庭满足的条件,具有很大的概率。因此,对于这个家庭,在参数空间的一个大的子集,恢复是数值稳定的。接下来,我们构造一个例子$\LSEM$的四个顶点与\emph{unbounded}条件数。然后,我们通过模拟以及现实世界的实验社会学应用证实了我们的理论研究结果。最后,我们提供了一个通用的启发式估计任何$\LSEM$实例的条件数。
Efficient Inference after Directionally Stable Adaptive Experiments
University College London(伦敦大学学院);Gatsby Computational Neuroscience Unit, University College London(伦敦大学学院计算神经科学单元);Cornell University(康奈尔大学);Netflix;Google DeepMind(谷歌DeepMind);Rutgers University(罗格斯大学)
MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference
Data Science & Artificial Intelligence Research Institute, China Unicom(中国unicom数据科学与人工智能研究 institute);Unicom Data Intelligence, China Unicom(中国unicom数据智能);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)