因果能否消除由相关性引起的混淆(在软件分析中)?
Can Causality Cure Confusion Caused By Correlation (in Software Analytics)?
AI总结:
本研究探讨在符号模型中引入因果意识分割标准是否能提升稳定性与鲁棒性,并比较人类与自动化模型的稳定性差异。
摘要:
背景:符号模型,特别是决策树,在软件工程中被广泛用于可解释性分析,如缺陷预测、配置调优和软件质量评估。大多数这些模型依赖于相关性分割标准,如方差减少或信息增益,这些标准识别统计关联,但无法推断X和Y之间的因果关系。最近的软件工程实证研究表明,相关性模型和因果发现算法都表现出显著的不稳定性。这种不稳定性源于两个互补的问题:1.基于相关性的方法混淆了关联与因果。2.因果发现算法依赖启发式近似来应对结构学习的NP难性质,导致其推断出的图在输入扰动下变化很大。两者共同削弱了现实世界SE任务中的信任、可重复性和解释的可靠性。目标:本研究探讨在符号模型中纳入因果意识的分割标准是否能提高其稳定性和鲁棒性,并且这些改进是否以预测或优化性能为代价。我们还检查了人类专家判断的稳定性与自动化模型的稳定性之间的比较。方法:使用MOOT存储库中的120多个多目标优化任务,我们通过预注册的bootstrap-ensemble协议评估稳定性,该协议通过胜利分数分配来测量方差。我们比较了人类因果评估的稳定性与基于相关性的决策树(EZR)。我们还将比较利用条件熵分割标准和混杂因子过滤的因果意识树。稳定性与性能差异通过统计方法(方差、基尼不纯度、KS检验、Cliff's delta)进行分析。
英文摘要:
Background: Symbolic models, particularly decision trees, are widely used in software engineering for explainable analytics in defect prediction, configuration tuning, and software quality assessment. Most of these models rely on correlational split criteria, such as variance reduction or information gain, which identify statistical associations but cannot imply causation between X and Y. Recent empirical studies in software engineering show that both correlational models and causal discovery algorithms suffer from pronounced instability. This instability arises from two complementary issues: 1-Correlation-based methods conflate association with causation. 2-Causal discovery algorithms rely on heuristic approximations to cope with the NP-hard nature of structure learning, causing their inferred graphs to vary widely under minor input perturbations. Together, these issues undermine trust, reproducibility, and the reliability of explanations in real-world SE tasks. Objective: This study investigates whether incorporating causality-aware split criteria into symbolic models can improve their stability and robustness, and whether such gains come at the cost of predictive or optimization performance. We additionally examine how the stability of human expert judgments compares to that of automated models. Method: Using 120+ multi-objective optimization tasks from the MOOT repository of multi-objective optimization tasks, we evaluate stability through a preregistered bootstrap-ensemble protocol that measures variance with win-score assignments. We compare the stability of human causal assessments with correlation-based decision trees (EZR). We would also compare the causality-aware trees, which leverage conditional-entropy split criteria and confounder filtering. Stability and performance differences are analyzed using statistical methods (variance, Gini Impurity, KS test, Cliff's delta)
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。