arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.03283cs.DBcs.CV
收藏

面向高效视觉数据查询处理的灵活扩展框架

Towards a Flexible Scale-out Framework for Efficient Visual Data Query Processing

Intel Labs(英特尔实验室)

Rohit Verma, Arun Raghunath

AI总结:

本文提出VDMS-Async,通过事件驱动架构和远程操作扩展,实现视觉数据查询的高效灵活处理,显著降低查询延迟并支持线性扩展。

摘要:

视觉数据管理系统正引起越来越多的关注,这些系统支持从图像调整大小到运行复杂机器学习模型等专门操作的查询。由于存在大量此类操作,尤其是在客户端希望在单个查询中运行多个此类操作时,以最短时间获得查询响应的基本需求受到了影响。现有系统采用一种临时方法,将不同的解决方案组合在一起,以提供端到端的视觉数据管理系统。与这类解决方案不同,视觉数据管理系统(VDMS)原生地执行具有多个操作的查询,从而提供了端到端的解决方案。然而,固定的原生操作子集和同步线程架构限制了其通用性和可扩展性。在本文中,我们开发了VDMS-Async,它增加了使用VDMS运行用户定义操作以及在远程服务器上执行查询内操作的能力。VDMS-Async利用事件驱动架构创建了一个高效的管道来执行查询内的操作。我们的实验表明,与现有最先进系统相比,VDMS-Async将查询执行时间减少了2-3倍。此外,远程操作与事件驱动架构相结合,使得VDMS-Async在每增加一台新的远程服务器时,查询执行时间能够线性扩展。我们展示了在添加64台远程服务器时,查询执行时间减少了64倍。

英文摘要:

There is growing interest in visual data management systems that support queries with specialized operations ranging from resizing an image to running complex machine learning models. With a plethora of such operations, the basic need to receive query responses in minimal time takes a hit, especially when the client desires to run multiple such operations in a single query. Existing systems provide an ad-hoc approach where different solutions are clubbed together to provide an end-to-end visual data management system. Unlike such solutions, the Visual Data Management System (VDMS) natively executes queries with multiple operations, thus providing an end-to-end solution. However, a fixed subset of native operations and a synchronous threading architecture limit its generality and scalability. In this paper, we develop VDMS-Async that adds the capability to run user-defined operations with VDMS and execute operations within a query on a remote server. VDMS-Async utilizes an event-driven architecture to create an efficient pipeline for executing operations within a query. Our experiments have shown that VDMS-Async reduces the query execution time by 2-3X compared to existing state-of-the-art systems. Further, remote operations coupled with an event-driven architecture enables VDMS-Async to scale query execution time linearly with the addition of every new remote server. We demonstrate a 64X reduction in query execution time when adding 64 remote servers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑