AI 中文总结
本研究通过挖掘GitHub上的28个开源鲁棒性工具仓库，发现其参与度与维护活动分布不均，多数仓库不活跃，呼吁将鲁棒性工具视为演进中的软件系统。
AI 中文摘要
鲁棒性评估对于将机器学习（ML）系统部署到真实场景中至关重要，在真实场景里模型可能面临对抗扰动、分布偏移及其他运行压力。包括Adversarial Robustness Toolbox、Foolbox和Robustness Gym在内的众多开源工具支持鲁棒性测试与评估。然而，尽管从业者可能依赖这些工具选择评估依赖项、复现鲁棒性评估并为AI保证提供证据，但人们对这些工具的维护、公众参与及长期可持续性知之甚少。我们对开源鲁棒性工具生态系统开展实证研究：从先前研究得出的精选种子集出发，系统搜索GitHub并识别出28个鲁棒性工具仓库；利用成熟软件工程指标分析仓库制品，以表征可观测的社区参与度、维护活动及项目寿命。结果显示，参与度与维护活动分布不均，持续活动集中在一小部分仓库中；在2026年1月21日的数据收集时点，5个仓库被归类为活跃、22个为不活跃、1个为已归档。这些发现强调需将鲁棒性工具视为不断演进的软件系统。
英文摘要
Robustness evaluation is essential for deploying machine-learning (ML) systems in real-world settings, where models may face adversarial perturbations, distribution shifts, and other operational stressors. Many open-source tools, including Adversarial Robustness Toolbox, Foolbox, and Robustness Gym, support robustness testing and evaluation. However, little is known about how these tools are maintained, publicly engaged with, and sustained over time, even though practitioners may rely on them to select evaluation dependencies, reproduce robustness assessments, and provide evidence for AI assurance. We present an empirical study of the open-source robustness tooling ecosystem. Starting from a curated seed set derived from prior work, we systematically searched GitHub and identified 28 robustness-tool repositories. We analyzed repository artifacts to characterize observable community engagement, maintenance activity, and project longevity using established software-engineering metrics. Our results show that engagement and maintenance are unevenly distributed, with sustained activity concentrated in a small subset of repositories. At the data collection date of January 21, 2026, five repositories were classified as active, 22 as inactive, and one as archived. These findings highlight the need to treat robustness tools as evolving software systems.
Comments32 pages. Preprint
