arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1505.01448cs.DCcs.OS
收藏

使用Robinhood策略引擎收回对HPC文件系统的控制

Taking back control of HPC file systems with Robinhood Policy Engine

Thomas Leibovici

摘要:

如今,最大的Lustre文件系统存储数十亿个条目。在这样的系统上,基于名称空间扫描的经典工具变得无法使用。诸如管理文件生存期、调度数据副本和生成整体文件系统统计信息等操作变得非常痛苦,因为它们需要收集、排序和聚合数十亿条记录的信息。Robinhood Policy Engine是一个开源软件,旨在解决这些挑战。它使得在大量文件系统条目上调度自动操作成为可能。它还通过提供有关数据所有权、年龄和大小配置文件的总体统计信息,对文件系统内容有了综合的了解。即使它可以与任何POSIX文件系统一起使用,Robinhood也支持Lustre特定的功能,如OST,池,HSM,ChangePath和DNE。它实现了对这些特性的特定支持,并利用它们来有效地管理Lustre文件系统。

英文摘要:

Today, the largest Lustre file systems store billions of entries. On such systems, classic tools based on namespace scanning become unusable. Operations such as managing file lifetime, scheduling data copies, and generating overall filesystem statistics become painful as they require collecting, sorting and aggregating information for billions of records. Robinhood Policy Engine is an open source software developed to address these challenges. It makes it possible to schedule automatic actions on huge numbers of filesystem entries. It also gives a synthetic understanding of file systems contents by providing overall statistics about data ownership, age and size profiles. Even if it can be used with any POSIX filesystem, Robinhood supports Lustre specific features like OSTs, pools, HSM, ChangeLogs, and DNE. It implements specific support for these features, and takes advantage of them to manage Lustre file systems efficiently.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑