我们在新“稀疏世界”中学到的十个教训：面向稀疏神经网络研究者的简短手册
Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
AI总结：
本文为稀疏神经网络研究者总结了十个常见问答，澄清了从稠密与稀疏到剪枝与稀疏训练等关键概念间的混淆，旨在为研究社区提供一般性知识和便利。
摘要：
本文不提出任何针对稀疏性的新算法或新硬件。相反，它旨在为日益繁荣的稀疏神经网络（SNN）研究社区服务“公共利益”。我们试图总结SNN中一些最常见的困惑，这些困惑可能出现在论文评审/反驳和报告等各种场景中——许多来自作者自己苦乐参半的经历！我们认为这样做是有意义且及时的，因为SNN研究的重点正显著从传统的剪枝转向训练前、训练中和训练后更多样化和更深刻的稀疏形式。它们的范围、假设和方法之间错综复杂的关系导致非专家甚至SNN专家产生误解。作为回应，我们从许多关键方面总结了关于SNN的十个问答，包括稠密与稀疏、非结构化稀疏与结构化稀疏、剪枝与稀疏训练、稠密到稀疏训练与稀疏到稀疏训练、静态稀疏与动态稀疏、训练前/训练中与训练后稀疏等等。我们努力提供恰当且普遍适用的答案，以尽可能澄清这些困惑。我们希望我们的总结为想进入并参与这个令人兴奋社区的人提供有用的一般知识；也为SNN研究者在正确的语境中解释他们的工作提供一些“安心”的便利。至少（也许作为本文最不重要的目标功能），如果你正在撰写/计划撰写SNN领域的论文或反驳，我们希望我们的某些答案能帮到你！
英文摘要：
This article does not propose any novel algorithm or new hardware for sparsity. Instead, it aims to serve the "common good" for the increasingly prosperous Sparse Neural Network (SNN) research community. We attempt to summarize some most common confusions in SNNs, that one may come across in various scenarios such as paper review/rebuttal and talks - many drawn from the authors' own bittersweet experiences! We feel that doing so is meaningful and timely, since the focus of SNN research is notably shifting from traditional pruning to more diverse and profound forms of sparsity before, during, and after training. The intricate relationships between their scopes, assumptions, and approaches lead to misunderstandings, for non-experts or even experts in SNNs. In response, we summarize ten Q\&As of SNNs from many key aspects, including dense vs. sparse, unstructured sparse vs. structured sparse, pruning vs. sparse training, dense-to-sparse training vs. sparse-to-sparse training, static sparsity vs. dynamic sparsity, before-training/during-training vs. post-training sparsity, and many more. We strive to provide proper and generically applicable answers to clarify those confusions to the best extent possible. We hope our summary provides useful general knowledge for people who want to enter and engage with this exciting community; and also provides some "mind of ease" convenience for SNN researchers to explain their work in the right contexts. At the very least (and perhaps as this article's most insignificant target functionality), if you are writing/planning to write a paper or rebuttal in the field of SNNs, we hope some of our answers could help you!
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
