压缩与比较:交互式评估机器学习模型压缩实验的效率与行为
Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments
AI总结:
本文提出交互式可视化系统Compress and Compare,通过统一界面跟踪压缩实验、比较模型预测权重激活,支持压缩分析任务,经案例研究和专家评估验证其提升效率与洞察力。
摘要:
为了在设备端部署机器学习模型,从业者使用压缩算法来缩小和加速模型,同时保持其高质量输出。压缩在实际应用中的一个关键方面是模型比较,包括跟踪许多压缩实验、识别模型行为中的细微变化,以及协商复杂的精度-效率权衡。然而,现有的压缩工具对比较的支持很差,导致分析繁琐且有时不完整,分散在不同的工具中。为了支持现实世界中的比较工作流程,我们开发了一个名为Compress and Compare的交互式可视化系统。在单一界面中,Compress and Compare通过可视化压缩模型之间的来源关系来展示有前景的压缩策略,并通过比较模型的预测、权重和激活来揭示压缩引起的行为变化。我们通过两个案例研究展示了Compress and Compare如何支持常见的压缩分析任务:调试生成语言模型上的失败压缩,以及识别图像分类模型中的压缩伪影。我们进一步在八位压缩专家的用户研究中评估了Compress and Compare,展示了其为压缩工作流程提供结构、帮助从业者建立压缩直觉,并鼓励对压缩对模型行为的影响进行彻底分析的潜力。通过这些评估,我们确定了未来可视化分析工具应考虑的压缩特定挑战,以及可能推广到更广泛模型比较任务的Compress and Compare可视化。
英文摘要:
To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex accuracy-efficiency trade-offs. However, existing compression tools poorly support comparison, leading to tedious and, sometimes, incomplete analyses spread across disjoint tools. To support real-world comparative workflows, we develop an interactive visual system called Compress and Compare. Within a single interface, Compress and Compare surfaces promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavior changes by comparing models' predictions, weights, and activations. We demonstrate how Compress and Compare supports common compression analysis tasks through two case studies, debugging failed compression on generative language models and identifying compression artifacts in image classification models. We further evaluate Compress and Compare in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help practitioners build intuition about compression, and encourage thorough analysis of compression's effect on model behavior. Through these evaluations, we identify compression-specific challenges that future visual analytics tools should consider and Compress and Compare visualizations that may generalize to broader model comparison tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。