增强用于可互操作联邦SPARQL评估的LargeRDFBench
Strengthening LargeRDFBench for Interoperable Federated SPARQL Evaluation
浏览论文内容
中文总结 AI 辅助
本研究修复了LargeRDFBench的数据质量问题，将其预期结果转为标准格式，还比较了FedX算法的源选择方式，增强了该基准的适用性与可复现性。
中文摘要 AI 辅助
LargeRDFBench是评估联邦SPARQL查询引擎最全面的基准之一，它结合了真实的关联数据集与丰富的查询套件，已成为该领域的参考标准。对联邦引擎的评估是通过将引擎结果与基准的预期结果进行比较来完成的，因此这些预期结果本身必须是可复现的。此外，该基准的多个数据转储违反了RDF规范，因此只有宽松解析RDF的引擎才能处理它们，且其预期结果以临时格式分发。我们通过可复现的清洗流水线识别、分类并修复了这些数据质量问题，为每个受影响的数据集生成了符合标准的序列化形式。我们还将基准的预期结果重新编码为W3C SPARQL 1.1查询结果JSON格式，并修正了其中的差异。现在所有数据集都能在严格的RDF解析器下解析，预期结果也可通过标准格式进行机器验证，这在忠实于原始数据的同时，将基准的适用范围扩展到了所有符合规范的引擎。通过独立实现端到端复现预期结果，我们发现了已发布参考中的损坏问题，以及我们的结果与原始结果之间的差异，其中一些差异并非易事，另一些则是未解决的问题。我们还首次对FedX算法中基于ASK和COUNT的源选择进行了初步比较。本研究通过使基准的产物与RDF标准保持一致，增强了这一已有价值的社区资源，扩大了可进行公平且可复现比较的引擎范围，同时提出了如何使自动源选择下的联邦查询结果可复现的问题。
英文摘要
LargeRDFBench is one of the most comprehensive benchmarks for evaluating federated SPARQL query engines, combining real, interlinked datasets with a rich query suite that has made it a reference point for the community. Evaluations of federated engines are published by comparing engine results against the benchmark's expected results, so those expected results must themselves be reproducible. Moreover, several of its data dumps violate the RDF specifications, so only engines that parse RDF leniently can host them, and its expected results are distributed in an ad hoc format. We identify, categorize and repair these data-quality issues with a reproducible cleaning pipeline, producing standards-conformant serializations of every affected dataset. Furthermore, we re-encode the benchmark's expected results in the W3C SPARQL 1.1 Query Results JSON Format and correct their discrepancies. Every dataset now parses under strict RDF parsers, and the expected results are machine-verifiable through a standard format, extending the benchmark's reach to the full range of conformant engines while staying faithful to the original data. Reproducing the expected results end-to-end with an independent implementation uncovers corruption in the published reference, and discrepancies between our results and the original ones, some not trivial to resolve, others open questions. We further perform a preliminary comparison, not previously explored, of ASK- and COUNT-based source selection in the FedX algorithm. This work strengthens an already valuable community resource by aligning its artifacts with the RDF standards, broadening the set of engines that can be fairly and reproducibly compared. We also raise the question of how the results of federated queries under automatic source selection can be made reproducible.
