OmniGenBench：为基因组基础模型自动化大规模计算机模拟基准测试
OmniGenBench: Automating Large-scale in-silico Benchmarking for Genomic Foundation Models
AI总结：
针对基因组基础模型缺乏统一基准与开源评测工具的问题，本文提出 GFMBench，整合四大基准、数百任务和数百万序列，自动化评测开源 GFM，并发布开源软件与公开排行榜。
摘要：
近年来，人工智能（如大语言模型，LLM）的进展推动了人们对基因组基础模型（GFM）取得突破的期待。自生命进化之初便隐藏在多样基因组中的“自然密码”，蕴含着通过基因组建模影响人类和生态系统的巨大潜力。近期 GFM 的突破（例如 Evo）为基因组建模吸引了大量投资与关注，因为它们解决了长期存在的挑战，并将计算机模拟基因组研究转变为自动化、可靠且高效的范式。在基因组学接连发生技术革命的繁荣时代背景下，GFM 研究面临两大挑战：缺乏 GFM 基准测试工具，以及缺少面向多样基因组学的开源软件。这些挑战阻碍了 GFM 的快速演进及其在理解和合成基因组等任务中的广泛应用，而这些问题已持续数十年。为应对这些挑战，我们提出 GFMBench，这是一个专注于面向 GFM 基准测试的框架。GFMBench 标准化了基准测试套件，并为大量开源 GFM 自动化执行基准测试。它整合了来自四个大规模基准、涵盖数百个基因组任务的数百万条基因组序列，使 GFM 能够普惠于广泛的计算机模拟基因组应用。此外，GFMBench 以开源软件形式发布，提供用户友好的接口和多样教程，适用于 AutoBench 以及 RNA 设计和结构预测等复杂任务。为促进基因组建模的进一步发展，我们推出了一个公开排行榜，展示由 AutoBench 得到的基准测试性能。GFMBench 代表着迈向标准化 GFM 基准测试和普惠化 GFM 应用的一步。
英文摘要：
The advancements in artificial intelligence in recent years, such as Large Language Models (LLMs), have fueled expectations for breakthroughs in genomic foundation models (GFMs). The code of nature, hidden in diverse genomes since the very beginning of life's evolution, holds immense potential for impacting humans and ecosystems through genome modeling. Recent breakthroughs in GFMs, such as Evo, have attracted significant investment and attention to genomic modeling, as they address long-standing challenges and transform in-silico genomic studies into automated, reliable, and efficient paradigms. In the context of this flourishing era of consecutive technological revolutions in genomics, GFM studies face two major challenges: the lack of GFM benchmarking tools and the absence of open-source software for diverse genomics. These challenges hinder the rapid evolution of GFMs and their wide application in tasks such as understanding and synthesizing genomes, problems that have persisted for decades. To address these challenges, we introduce GFMBench, a framework dedicated to GFM-oriented benchmarking. GFMBench standardizes benchmark suites and automates benchmarking for a wide range of open-source GFMs. It integrates millions of genomic sequences across hundreds of genomic tasks from four large-scale benchmarks, democratizing GFMs for a wide range of in-silico genomic applications. Additionally, GFMBench is released as open-source software, offering user-friendly interfaces and diverse tutorials, applicable for AutoBench and complex tasks like RNA design and structure prediction. To facilitate further advancements in genome modeling, we have launched a public leaderboard showcasing the benchmark performance derived from AutoBench. GFMBench represents a step toward standardizing GFM benchmarking and democratizing GFM applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
