Unix工具与FITO类别错误:崩溃一致性与持久性的协议性质
Unix Tools and the FITO Category Mistake: Crash Consistency and the Protocol Nature of Persistence
AI总结:
Unix工具的文件系统抽象存在虚假的原子性假设,导致存储堆栈中非原子依赖链及严重系统故障。
摘要:
Unix工具如ls、cp、mv和rename暴露了一个文件系统抽象,似乎呈现一个单一的、权威的状态通过原子转换演变。这个抽象是虚假的。我们提出了一种系统性的向前时间-only(FITO)分析,证明假设瞬间的原子状态转换在计算堆栈的每一个层次上都构成了类别错误——从ext4日志记录和延迟分配,通过fsync失败语义、NVMe Flush/FUA设备行为,以及Linux可重启序列,下至x86-64CPU自身无法保证在不可屏蔽中断下原子性进入监督模式。我们证明了一个形式上的不可能结果:没有任何基于系统调用的持久性原语可以在失败情况下定义提交边界,因为系统调用返回值在多个实质性不同的持久性状态之间保持一致。我们识别出跨层时间假设泄漏是类别错误传播的结构性机制,并展示整个存储堆栈形成了一条递归的非原子依赖链,其表面上的原子性反映的是数学上的不可能性(Herlihy,1991),而非仅仅是工程缺陷。附录记录了现实中的后果:由重试放大驱动的谷歌、AWS、Meta和Cloudflare的级联云故障;PostgreSQL、etcd和MySQL中因fsync失败导致的数据库损坏;CERN、NetApp和Meta中的静默数据损坏;在大规模AI训练中消耗12-43%的计算预算;以及每年总金额达数十亿美元的金融系统故障。这些后果追溯到单一结构性原因:系统围绕FITO假设设计,通过重试和恢复协议来补偿其失败,这些协议放大了它们试图掩盖的失败。
英文摘要:
Unix tools such as ls, cp, mv, and rename expose a filesystem abstraction that appears to present a single, authoritative state evolving through atomic transitions. This abstraction is false. We present a systematic Forward-In-Time-Only (FITO) analysis demonstrating that the assumption of instantaneous atomic state transitions constitutes a category mistake at every layer of the computing stack -- from ext4 journaling and delayed allocation, through fsync failure semantics, NVMe Flush/FUA device behavior, and Linux restartable sequences, down to the x86-64 CPU's own inability to guarantee atomic supervisor entry under Non-Maskable Interrupts. We prove a formal impossibility result: no syscall-based persistence primitive can define a commit boundary under failure, because the syscall return value is consistent with multiple materially different persistence states across Linux filesystems. We identify cross-layer temporal assumption leakage as the structural mechanism by which the category mistake propagates, and show that the entire storage stack forms a recursive chain of non-atomic dependencies whose apparent atomicity reflects mathematical impossibility (Herlihy, 1991), not merely engineering deficiency. An appendix documents the real-world consequences: cascading cloud outages at Google, AWS, Meta, and Cloudflare driven by retry amplification; database corruption from fsync failures in PostgreSQL, etcd, and MySQL; silent data corruption at CERN, NetApp, and Meta; AI training waste consuming 12--43% of compute budgets at scale; and financial system failures totaling billions of dollars annually. These consequences trace to a single structural cause: systems designed around the FITO assumption, compensating for its failure with retry-and-recover protocols that amplify the very failures they attempt to mask.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。