arXiv 2302.080072023-02cs.LGcs.AIcs.AR

共享微指数：轻微移位，效果显著

With Shared Microexponents, A Little Shifting Goes a Long Way

Microsoft（微软）；Meta

Bita Rouhani, Ritchie Zhao, Venmugil Elango, Rasoul Shafipour, Mathew Hall, Maral Mesmakhosroshahi, Ankit More, Levi Melnick, Maximilian Golub, Girish Varatkar,… 展开作者

Bita Rouhani, Ritchie Zhao, Venmugil Elango, Rasoul Shafipour, Mathew Hall, Maral Mesmakhosroshahi, Ankit More, Levi Melnick, Maximilian Golub, Girish Varatkar, Lei Shao, Gaurav Kolhe, Dimitry Melts, Jasmine Klar, Renee L'Heureux, Matt Perry, Doug Burger, Eric Chung, Zhaoxia Deng, Sam Naghshineh, Jongsoo Park, Maxim Naumov

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出块数据表示框架，发现基于共享微指数的新格式 MX，其量化效果优于现有窄精度浮点和块浮点方法，并在大规模生成模型和推荐系统中验证有效。

摘要：

本文提出块数据表示（Block Data Representations，BDR）框架，用于探索和评估面向深度学习的多种窄精度格式。该框架支持对主流量化标准进行比较，并借助 BDR 识别出基于共享微指数（MX）的新格式，其性能优于其他最先进的量化方法，包括窄精度浮点和块浮点。MX 在硬件中利用基于共享微指数的超细缩放因子，实现多级量化缩放。MX 的有效性在真实世界模型上得到验证，包括大规模生成式预训练与推理以及生产级推荐系统。

英文摘要：

This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-point and block floating-point. MX utilizes multiple levels of quantization scaling with ultra-fine scaling factors based on shared microexponents in the hardware. The effectiveness of MX is demonstrated on real-world models including large-scale generative pretraining and inferencing, and production-scale recommendation systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。