RepTran：基于搜索的Transformer模型修复
RepTran: Search-Based Repair of Transformer Models
另 1 家 · 查看机构详情
- The University of Osaka, Japan（大阪大学）
- National Institute of Informatics, Japan（日本信息处理研究所）
- Kyushu University, Japan（九州大学）
- Waseda University, Japan（早稻田大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
研究针对Transformer模型行为不当问题，提出RepTran基于搜索的修复方法，结合两种分数识别可疑权重，用差分进化迭代优化，经实验对比多个基线，该方法平均修复率达74.7%，有效提升了人工智能软件可靠性。
中文摘要 AI 辅助
为确保人工智能软件的整体质量，不仅传统软件组件，人工智能组件也需测试和修复。Transformer模型在软件系统中愈发重要，其行为不当影响重大。此前软件工程领域虽提出深度神经网络修复方法，但多忽视Transformer特定结构。本文提出RepTran，一种针对Transformer模型的基于搜索的修复方法，针对其前馈网络，通过结合基于方差的神经元分数和现有双向分数识别可疑权重，再用差分进化迭代优化权重。实验采用由CIFAR - 100和Tiny - ImageNet构建的18个故障基准，与随机权重选择、Arachne及ArachneW对比。RepTran平均修复率达74.7%，在所有基准测试中统计上优于随机选择和Arachne，无论所选权重数量，其修复率均高于ArachneW，表明该方法对提高人工智能软件可靠性有效。
英文摘要
To ensure the overall quality of AI-enabled software, not only traditional software components but also AI components need to be tested and repaired. Among AI components, Transformer models are increasingly integrated into software systems, which makes their misbehaviors critical. Although prior work in the software engineering community has proposed deep neural network (DNN) repair methods, most overlook Transformer-specific structures. We propose RepTran, a search-based repair method for Transformer models. It targets their feed-forward networks (FFNs), which play a central role in the architecture. RepTran identifies suspicious weights by combining two types of scores: a variance-based neuron score and an existing bidirectional score. It then iteratively optimizes these weights using differential evolution. Our evaluation includes 18 fault benchmarks constructed from CIFAR-100 and Tiny-ImageNet. We compare RepTran against three baselines: random weight selection, Arachne (a state-of-the-art DNN repair method), and ArachneW, which enables Arachne to control the number of selected weights. RepTran achieved an average repair rate of 74.7%, statistically outperforming random selection and Arachne across all benchmarks. Effect size analysis revealed that RepTran achieved higher repair rates than ArachneW regardless of the number of selected weights. These results suggest that RepTran is effective for enhancing the reliability of AI-enabled software.
