arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.03493q-bio.MNcs.LG
收藏

在基因调控网络基准测试中量化评估协议轴上的排名不稳定性

Quantifying Ranking Instability Across Evaluation Protocol Axes in Gene Regulatory Network Benchmarking

Department of Computer Science University of Tübingen(计算机科学系图宾根大学)

Ihor Kendiukhov

AI总结:

本研究提出了一种系统诊断框架,用于量化基因调控网络基准测试中评估协议轴上的排名不稳定性,并揭示了方法相对区分能力变化对反转率的影响。

摘要:

在基因调控网络(GRN)推断中,排名通常用于支持关于方法质量的科学结论,但这些排名在合理评估协议选择下的稳定性很少被检查。我们提出了一种系统诊断框架,用于测量在协议转移下的排名不稳定性,包括分解工具,用于将基础率效应与区分效应分开。利用现有单细胞GRN基准输出,涵盖三种人类组织和六种推断方法,我们量化了四个协议轴上的成对反转率:候选集限制(16.3%,95%置信区间11.0至23.4%)、组织背景(19.3%)、参考网络选择(32.1%)和符号映射政策(0.0%)。一个排列空假设证实,观察到的反转率远低于随机顺序期望(0.163 versus null均值0.500),表明部分稳定但非不变的排名结构。我们的分解显示,反转是由方法相对区分能力的变化而非基础率膨胀驱动的,这一发现挑战了GRN基准测试中常见的隐含假设。我们提出了针对稳定性意识评估的具体报告实践,并提供了一种诊断工具包,用于识别可能产生反转的方法对。

英文摘要:

Benchmark rankings are routinely used to justify scientific claims about method quality in gene regulatory network (GRN) inference, yet the stability of these rankings under plausible evaluation protocol choices is rarely examined. We present a systematic diagnostic framework for measuring ranking instability under protocol shift, including decomposition tools that separate base rate effects from discrimination effects. Using existing single cell GRN benchmark outputs across three human tissues and six inference methods, we quantify pairwise reversal rates across four protocol axes: candidate set restriction (16.3 percent, 95 percent CI 11.0 to 23.4 percent), tissue context (19.3 percent), reference network choice (32.1 percent), and symbol mapping policy (0.0 percent). A permutation null confirms that observed reversal rates are far below random order expectations (0.163 versus null mean 0.500), indicating partially stable but non invariant ranking structure. Our decomposition reveals that reversals are driven by changes in the relative discrimination ability of methods rather than by base rate inflation, a finding that challenges a common implicit assumption in GRN benchmarking. We propose concrete reporting practices for stability aware evaluation and provide a diagnostic toolkit for identifying method pairs at risk of reversal.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑