多语言开发者的困境:对十二种规范在对应数量语言中的一致性测试
The Polyglot's Dilemma: Conformance Testing a Dozen Specs in as Many Languages
中文总结 AI 辅助
针对MongoDB十二种原生实现的客户端库一致性测试难题,提出基于YAML的统一测试格式,删除大量测试代码并降低了不一致缺陷发生率,同时总结了相关经验教训。
中文摘要 AI 辅助
MongoDB维护着十二种编程语言的客户端库,被数万个组织和数百万开发者使用,多数以原生方式实现而非共享核心的封装。确保这些由数百万行代码组成的库行为一致,难度大却至关重要。十一年来,我们开发了一种基于规范的测试方法:用YAML编写一次测试,再由各库的特定语言解释器执行。我们描述了从多种临时格式向统一测试格式的演进,这让我们删除了超过22000行测试代码。采用YAML测试的驱动程序中,不一致缺陷的发生率下降了86%(尽管结果存在差异)。我们报告了在声明式测试设计、测试架构、模式演进以及统一的局限性方面的经验教训。
英文摘要
MongoDB maintains client libraries in a dozen programming languages, used by tens of thousands of organizations and millions of developers. Most are implemented natively rather than as wrappers around a shared core. Ensuring consistent behavior across these libraries, comprising millions of lines of code, is hard but essential. Over eleven years, we developed a specification-based testing approach: tests are written once in YAML and executed by language-specific interpreters for each library. We describe the evolution from many ad-hoc formats to a Unified Test Format, which allowed us to delete over 22,000 lines of test code. The rate of nonconformance bugs fell up to 86% in drivers that adopted YAML tests (though results varied). We report lessons learned about declarative test design, test architecture, schema evolution, and the limits of unification.