arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.06361cs.LGq-bio.QM
收藏

DeepGOPlus推理的数值稳定性

Numerical Stability of DeepGOPlus Inference

Concordia University(康考迪亚大学);Child Mind Institute(儿童心理研究所)

Inés Gonzalez Pepe, Yohan Chatelain, Gregory Kiar, Tristan Glatard

AI总结:

本文通过Monte Carlo Arithmetic与VPREC评估蛋白质功能预测模型DeepGOPlus推理阶段的浮点扰动稳定性及降精度可行性,证明其数值可靠但低精度部署需选择性进行。

摘要:

卷积神经网络(CNN)目前是应用最广泛的深度神经网络(DNN)架构之一,并在许多问题上取得了最先进的性能。CNN最初应用于计算机视觉任务,但除图像外,它也能很好地处理任何具有空间关系的数据,并已被应用于不同领域。然而,近期研究强调了DNN中的数值稳定性挑战,这也与其已知的对噪声注入的敏感性有关。这些挑战可能危及DNN的性能和可靠性。本文研究了用于预测蛋白质功能的CNN——DeepGOPlus。DeepGOPlus已取得最先进的性能,并能够成功利用和注释蛋白质组学中大量涌现的蛋白质序列。我们通过量化由底层浮点数据扰动产生的数值不确定性,来确定该模型推理阶段的数值稳定性。此外,我们探索了在DeepGOPlus推理中使用降精度浮点格式以减少内存消耗和延迟的机会。这是通过使用Monte Carlo Arithmetic(一种通过实验量化浮点运算误差的技术)和VPREC(一种以可自定义浮点精度格式模拟结果的工具)对DeepGOPlus的执行进行插桩来实现的。研究重点放在推理阶段,因为它是DeepGOPlus模型的主要交付物,可在不同环境中广泛应用。总体而言,我们的结果表明,尽管DeepGOPlus CNN在数值上非常稳定,但它只能选择性地以较低精度浮点格式实现。我们得出结论:从预训练DeepGOPlus模型获得的预测在数值上非常可靠,并且高效地使用了现有浮点格式。

英文摘要:

Convolutional neural networks (CNNs) are currently among the most widely-used deep neural network (DNN) architectures available and achieve state-of-the-art performance for many problems. Originally applied to computer vision tasks, CNNs work well with any data with a spatial relationship, besides images, and have been applied to different fields. However, recent works have highlighted numerical stability challenges in DNNs, which also relates to their known sensitivity to noise injection. These challenges can jeopardise their performance and reliability. This paper investigates DeepGOPlus, a CNN that predicts protein function. DeepGOPlus has achieved state-of-the-art performance and can successfully take advantage and annotate the abounding protein sequences emerging in proteomics. We determine the numerical stability of the model's inference stage by quantifying the numerical uncertainty resulting from perturbations of the underlying floating-point data. In addition, we explore the opportunity to use reduced-precision floating point formats for DeepGOPlus inference, to reduce memory consumption and latency. This is achieved by instrumenting DeepGOPlus' execution using Monte Carlo Arithmetic, a technique that experimentally quantifies floating point operation errors and VPREC, a tool that emulates results with customizable floating point precision formats. Focus is placed on the inference stage as it is the primary deliverable of the DeepGOPlus model, widely applicable across different environments. All in all, our results show that although the DeepGOPlus CNN is very stable numerically, it can only be selectively implemented with lower-precision floating-point formats. We conclude that predictions obtained from the pre-trained DeepGOPlus model are very reliable numerically, and use existing floating-point formats efficiently.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑