COPR——高效的大规模日志存储与检索
COPR -- Efficient, large-scale log storage and retrieval
AI总结：
针对大规模流式日志数据的存储与检索问题，提出 Compressed Probabilistic Retrieval 算法（COPR）作为现有索引结构的替代方案，在大幅降低存储空间和假阳性率的同时显著提升了查询吞吐量。
摘要：
现代大规模监控系统必须在近实时条件下处理并存储海量日志数据。在查询时，系统必须能够基于日志消息的内容找到相关日志，同时使用能够扩展至该数据量级且仍保持高效可用的支持结构。我们提出了一种新颖的 Compressed Probabilistic Retrieval 算法（COPR），它能够回答 Multi-Set Multi-Membership-Queries，可用作流式日志数据现有索引结构的替代方案。在实验中，与所测试的最先进的倒排索引相比，COPR 所需的存储空间最多减少了 93%；与所测试的最先进的成员关系草图相比，其假阳性率最多低了四个数量级。此外，COPR 的查询吞吐量比所测试的倒排索引最高达 250 倍，比所测试的成员关系草图最高达 240 倍。
英文摘要：
Modern, large scale monitoring systems have to process and store vast amounts of log data in near real-time. At query time the systems have to find relevant logs based on the content of the log message using support structures that can scale to these amounts of data while still being efficient to use. We present our novel Compressed Probabilistic Retrieval algorithm (COPR), capable of answering Multi-Set Multi-Membership-Queries, that can be used as an alternative to existing indexing structures for streamed log data. In our experiments, COPR required up to 93% less storage space than the tested state-of-the-art inverted index and had up to four orders of magnitude less false-positives than the tested state-of-the-art membership sketch. Additionally, COPR achieved up to 250 times higher query throughput than the tested inverted index and up to 240 times higher query throughput than the tested membership sketch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
