arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.06975cs.DCcs.SE
收藏

TraceMesh:面向分布式追踪的可扩展流式采样方法

TraceMesh: Scalable and Streaming Sampling for Distributed Traces

Zhuangbin Chen, Zhihan Jiang, Yuxin Su, Michael R. Lyu, Zibin Zheng

AI总结:

TraceMesh提出一种基于局部敏感哈希和演化聚类的可扩展流式分布式追踪采样方法,通过低维投影与动态采样决策避免冗余,在采样准确性和效率上显著优于现有方法。

摘要:

分布式追踪是云基与数据中心系统监控中的基础要素。它能够提供请求或操作跨多个服务的全生命周期可见性,这对于理解系统依赖关系和性能瓶颈至关重要。为了降低计算与存储开销,大多数追踪框架采用统一采样策略,这不可避免地会采集到重叠且冗余的信息。更先进的方法采用基于学习的方式,使采样偏向于更具信息量的追踪。然而,现有方法未能充分考虑追踪数据的高维性和动态性,而这对于追踪采样在生产环境中的部署至关重要。为了解决这些实际挑战,本文提出了TraceMesh,一种面向分布式追踪的可扩展流式采样器。TraceMesh利用局部敏感哈希(Locality-Sensitivity Hashing,LSH),通过将追踪投影到低维空间并保持其相似性来提高采样效率。在此过程中,TraceMesh以统一且简化的方式容纳先前未见过的追踪特征。随后,TraceMesh通过演化聚类对追踪进行采样,动态调整采样决策以避免对重复出现的追踪过度采样。所提出的方法使用从开源微服务基准和生产服务系统收集的追踪数据进行了评估。实验结果表明,TraceMesh在采样准确性和效率方面均显著优于最先进的方法。

英文摘要:

Distributed tracing serves as a fundamental element in the monitoring of cloud-based and datacenter systems. It provides visibility into the full lifecycle of a request or operation across multiple services, which is essential for understanding system dependencies and performance bottlenecks. To mitigate computational and storage overheads, most tracing frameworks adopt a uniform sampling strategy, which inevitably captures overlapping and redundant information. More advanced methods employ learning-based approaches to bias the sampling toward more informative traces. However, existing methods fall short of considering the high-dimensional and dynamic nature of trace data, which is essential for the production deployment of trace sampling. To address these practical challenges, in this paper we present TraceMesh, a scalable and streaming sampler for distributed traces. TraceMesh employs Locality-Sensitivity Hashing (LSH) to improve sampling efficiency by projecting traces into a low-dimensional space while preserving their similarity. In this process, TraceMesh accommodates previously unseen trace features in a unified and streamlined way. Subsequently, TraceMesh samples traces through evolving clustering, which dynamically adjusts the sampling decision to avoid over-sampling of recurring traces. The proposed method is evaluated with trace data collected from both open-source microservice benchmarks and production service systems. Experimental results demonstrate that TraceMesh outperforms state-of-the-art methods by a significant margin in both sampling accuracy and efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑