首页/论文/2001.09078

arXiv 2001.090782020-01-24cs.DB

Adaptive Low-level Storage of Very Large Knowledge Graphs

Jacopo Urbani, Ceriel Jacobs

备注：: Accepted WWW 2020

英文摘要：

The increasing availability and usage of Knowledge Graphs (KGs) on the Web calls for scalable and general-purpose solutions to store this type of data structures. We propose Trident, a novel storage architecture for very large KGs on centralized systems. Trident uses several interlinked data structures to provide fast access to nodes and edges, with the physical storage changing depending on the topology of the graph to reduce the memory footprint. In contrast to single architectures designed for single tasks, our approach offers an interface with few low-level and general-purpose primitives that can be used to implement tasks like SPARQL query answering, reasoning, or graph analytics. Our experiments show that Trident can handle graphs with 10^11 edges using inexpensive hardware, delivering competitive performance on multiple workloads.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

RodentStore，一种自适应声明式存储系统的案例

The Case for RodentStore, an Adaptive, Declarative Storage System

最近数据库社区围绕新应用程序的兴奋？分析的、科学的、图表的、地理空间的等等。导致了数据库存储系统研究的爆炸式增长。新的存储系统对数据库社区至关重要，因为它们是使数据库系统在新的应用领域中表现良好的核心。不幸的是，每个这样的系统也代表了大量的工程努力，包括大量的重复机制的功能，如事务和缓存。在本文中，我们的情况下，RodentStore，自适应和声明性存储系统提供了一个高层次的接口，用于描述数据的物理表示。具体来说，RodentStore使用声明式存储代数，管理员（或数据库设计工具）可以指定如何将逻辑模式分组为行、列和/或数组的集合，以及这些组在磁盘上的布局顺序。我们描述了我们的代数的关键运算符和类型，概述了RodentStore的一般架构，它解释代数表达式来生成数据的物理表示，并描述了RodentStore和数据库系统的其他部分，如查询优化器和执行器之间的接口。我们提供了一个案例研究的RodentStore在代表密集的地理空间数据收集的移动传感器网络中的潜在用途，显示不同的存储布局可以表示使用我们的一些代数结构和潜在的性能增益，一个RodentStore内置的存储系统可以提供的方便。

2009-09-09cs.DB

Breaking Down Memory Walls: Adaptive Memory Management in LSM-based Storage Systems (Extended Version)

Log-Structured Merge-trees (LSM-trees) have been widely used in modern NoSQL systems. Due to their out-of-place update design, LSM-trees have introduced memory walls among the memory components of multiple LSM-trees and between the write memory and the buffer cache. Optimal memory allocation among these regions is non-trivial because it is highly workload-dependent. Existing LSM-tree implementations instead adopt static memory allocation schemes due to their simplicity and robustness, sacrificing performance. In this paper, we attempt to break down these memory walls in LSM-based storage syste…

2020-07-14cs.DB

Lance：通过自适应结构编码实现列式存储中的高效随机访问

Lance: Efficient Random Access in Columnar Storage through Adaptive Structural Encodings

本文针对NVMe存储下列式格式随机访问性能不足的问题，提出Lance自适应结构编码方案，通过交替两种编码实现高效随机访问，优于Parquet且无扫描或内存权衡。

2025-04-21cs.DB

优化的磁盘布局用于交互图的自适应存储

Optimized Disk Layouts for Adaptive Storage of Interaction Graphs

我们生活在一个联系日益紧密的世界中，记录人、软件系统和物理世界之间交互的数据变得越来越普遍。这些数据通常采用时间演化图的形式，其中实体是顶点，它们之间的交互是边。我们称这样的图为交互图。包括电信、交通和社交媒体在内的各种应用领域都依赖于在交互图上执行的分析。要有效地支持交互图上的历史分析，就需要针对磁盘上的数据布局问题提供有效的解决方案。本文提出了一种自适应磁盘布局称为铁路布局优化磁盘块存储的相互作用图。关键思想是将块划分为一个或多个子块，其中每个子块包含属性的子集，但整个图结构在每个子块内复制。这改进了查询I/O，但代价是增加了存储开销。我们介绍了最佳的ILP配方分区磁盘块成子块重叠和非重叠属性。此外，我们提出了贪婪的启发式方法，可以更好地扩展相比，ILP的替代品，但实现接近最佳的查询I/O。为了证明铁路布局的好处，我们提供了一个广泛的实验研究比较我们的方法，一些基线的替代品。

2014-10-20cs.DB

Towards Adaptive Storage Views in Virtual Memory

Traditionally, DBMSs separate their storage layer from their indexing layer. While the storage layer physically materializes the database and provides low-level access methods to it, the indexing layer on top enables a faster locating of searched-for entries. While this clearly separates concerns, it also adds a level of indirection to the already complex execution path. In this work, we propose an alternative design: Instead of conservatively separating both layers, we naturally fuse them by integrating an adaptive coarse-granular indexing scheme directly into the storage layer. We do so by u…

2022-12-06cs.DBcs.OS

Efficient RDF Graph Storage based on Reinforcement Learning

Knowledge graph is an important cornerstone of artificial intelligence. The construction and release of large-scale knowledge graphs in various fields pose new challenges to knowledge graph data management. Due to the maturity and stability, relational database is also suitable for RDF data storage. However, the complex structure of RDF graph brings challenges to storage structure design for RDF graph in the relational database. To address the difficult problem, this paper adopts reinforcement learning (RL) to optimize the storage partition method of RDF graph based on the relational database.…

2020-10-22cs.DB

更多相关论文收起更多论文

WikiKV: 面向层次知识导航的架构演化路径索引存储

WikiKV: Schema-Evolving Path-Indexed Storage for Hierarchical Knowledge Navigation

Tencent Inc.（腾讯公司）

提出WikiKV，一种专为LLM策展层次知识库设计的路径索引键值存储模型，通过数据驱动架构、一致性协议和预算导航算子，实现低延迟查询和渐进式答案优化。

2026-06-15cs.DB

EKO: Adaptive Sampling of Compressed Video Data

Researchers have presented systems for efficiently analysing video data at scale using sampling algorithms. While these systems effectively leverage the temporal redundancy present in videos, they suffer from three limitations. First, they use traditional video storage formats are tailored for human consumption. Second, they load and decode the entire compressed video in memory before applying the sampling algorithm. Third, the sampling algorithms often require labeled training data obtained using a specific deep learning model. These limitations lead to lower accuracy, higher query execution …

2021-04-04cs.DB

A Dual-Store Structure for Knowledge Graphs

To effectively manage increasing knowledge graphs in various domains, a hot research topic, knowledge graph storage management, has emerged. Existing methods are classified to relational stores and native graph stores. Relational stores are able to store large-scale knowledge graphs and convenient in updating knowledge, but the query performance weakens obviously when the selectivity of a knowledge graph query is large. Native graph stores are efficient in processing complex knowledge graph queries due to its index-free adjacent property, but they are inapplicable to manage a large-scale knowl…

2020-12-13cs.DB

关联知识图谱用于高效序列存储与检索

Associative Knowledge Graphs for Efficient Sequence Storage and Retrieval

Institute of Management and Information Technology（管理与信息技术研究所）；University of Information Technology and Management（信息科技与管理大学）；Silesian University of Technology（桑特大学技术学院）；AGH University of Krakow（克拉科夫AGH大学）；Rzeszów University of Technology（瑞兹大学技术学院）

本文提出基于关联知识图谱（AKGs）的序列存储与检索新方法，通过顺序结构关联知识图谱（SSAKGs）编码序列，开发四种排序算法，在合成和真实数据集上验证了高内存容量、无需训练及上下文重建的高效性。

2025-09-09cs.AIcs.DB

面向高性能计算系统中可扩展知识图谱驱动的运维数据分析的统一本体

A Unified Ontology for Scalable Knowledge Graph-Driven Operational Data Analytics in High-Performance Computing Systems

本文提出首个面向HPC运维数据分析的统一本体，建模两大公开数据集，通过36个胜任力问题验证，并优化知识图谱存储开销最高达38.84%，支持跨系统分析。

2025-07-10cs.DBcs.DC

REIS：一种高性能且高能效的存储内处理检索系统

REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing

ETH Zürich（苏黎世联邦理工学院）；King’s College London（伦敦国王学院）；POSTECH

针对RAG检索阶段因ANNS数据移动开销导致的瓶颈，提出首个面向RAG的ISP系统REIS，通过定制数据库布局、数据放置技术及利用存储内部计算资源，相比服务器级系统平均提升13倍检索性能和55倍能效。

2025-11-11cs.ARcs.CLcs.DB

AWAPart: Adaptive Workload-Aware Partitioning of Knowledge Graphs

University of Georgia（佐治亚大学）

Large-scale knowledge graphs are increasingly common in many domains. Their large sizes often exceed the limits of systems storing the graphs in a centralized data store, especially if placed in main memory. To overcome this, large knowledge graphs need to be partitioned into multiple sub-graphs and placed in nodes in a distributed system. But querying these fragmented sub-graphs poses new challenges, such as increased communication costs, due to distributed joins involving cut edges. To combat these problems, a good partitioning should reduce the edge cuts while considering a given query work…

2022-03-28cs.AIcs.DBcs.DC