arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.23220cs.DC
收藏

STELLAR:利用LLM自主推理的存储调优引擎用于高性能并行文件系统

STELLAR: Storage Tuning Engine Leveraging LLM Autonomous Reasoning for High Performance Parallel File Systems

University of Delaware(特拉华大学);Argonne National Laboratory(阿贡国家实验室)

Chris Egersdoerfer, Philip Carns, Shane Snyder, Robert Ross, Dong Dai

AI总结:

STELLAR利用LLM自主推理实现高性能并行文件系统的存储调优,通过多代理设计和检索增强生成技术提升调优效率与准确性。

摘要:

I/O性能对于数据密集型科学计算的效率至关重要;但调优大规模存储系统是复杂的、昂贵的,并且 notoriously 人力密集,使得大多数领域科学家无法接触。为了解决这个问题,我们提出了STELLAR,一种用于高性能并行文件系统的自主调优器。我们的评估显示,STELLAR几乎总是能在前五次尝试中为并行文件系统选择接近最优的参数配置,即使对于以前未见过的应用也是如此。STELLAR与传统自动调优方法有根本的不同,后者通常需要数万次迭代才能收敛。借助大型语言模型(LLMs),STELLAR通过(1)准确地从软件手册中提取可调参数,(2)分析由应用程序生成的I/O跟踪日志,(3)选择初始调优策略,(4)在真实系统上重新运行应用程序并收集I/O性能反馈,(5)调整调优策略并重复调优循环,(6)反思和总结调优经验,将其转化为可重用的知识用于未来优化。STELLAR集成了检索增强生成(RAG)、工具执行、基于LLM的推理和多代理设计,以稳定推理并对抗幻觉。我们评估了每个组件对优化结果的影响,为其他优化领域的类似系统提供了设计见解。STELLAR的架构和实证结果突显了一种有前景的复杂系统优化方法,特别是对于具有大搜索空间和高探索成本的问题,同时使I/O调优对具有最小附加资源的领域科学家更加可及。

英文摘要:

I/O performance is crucial to efficiency in data-intensive scientific computing; but tuning large-scale storage systems is complex, costly, and notoriously manpower-intensive, making it inaccessible for most domain scientists. To address this problem, we propose STELLAR, an autonomous tuner for high-performance parallel file systems. Our evaluations show that STELLAR almost always selects near-optimal parameter configurations for parallel file systems within the first five attempts, even for previously unseen applications. STELLAR differs fundamentally from traditional autotuning methods, which often require hundreds of thousands of iterations to converge. Powered by large language models (LLMs), STELLAR enables autonomous end-to-end agentic tuning by (1) accurately extracting tunable parameters from software manuals, (2) analyzing I/O trace logs generated by applications, (3) selecting initial tuning strategies, (4) rerunning applications on real systems and collecting I/O performance feedback, (5) adjusting tuning strategies and repeating the tuning cycle, and (6) reflecting on and summarizing tuning experiences into reusable knowledge for future optimizations. STELLAR integrates retrieval-augmented generation (RAG), tool execution, LLM-based reasoning, and a multiagent design to stabilize reasoning and combat hallucinations. We evaluate the impact of each component on optimization outcomes, providing design insights for similar systems in other optimization domains. STELLAR's architecture and empirical results highlight a promising approach to complex system optimization, especially for problems with large search spaces and high exploration costs, while making I/O tuning more accessible to domain scientists with minimal added resources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑