从线程到跨大陆计算机：分布式超级计算中令人不安的教训
From Thread to Transcontinental Computer: Disturbing Lessons in Distributed Supercomputing
摘要：
我们描述了天文CosmoGrid项目期间遇到的政治和技术的并发症。CosmoGrid是对宇宙中大尺度结构形成的数值研究。由于空间和时间坐标的巨大动态范围以及所需的巨大计算机资源，模拟具有挑战性。在CosmoGrid中，我们通过光网络连接多达四台超级计算机来处理计算需求，并使它们作为一台机器运行。这是具有挑战性的，如果只是因为我们选择的超级计算机被半个地球分开，因为其中三个分散在欧洲，第四个在东京。多台计算机的协同调度和代码的“网格化”使我们能够为这台分布式洲际超级计算机实现高达$93\%$的效率。在这项工作中，我们发现，网格上的高性能计算可以更有效地完成，如果所涉及的网站愿意灵活的用户策略，并有设施，以提供这种灵活性可能是关键，以加强HPC社区的地位，在日益云主导的计算环境。考虑到研究小组或大学部门拥有的较小的计算机集群通常具有灵活的用户策略，我们认为，通过组合数十个，数百个甚至数千个这些资源来实现分布式超级计算可能更容易。
英文摘要：
We describe the political and technical complications encountered during the astronomical CosmoGrid project. CosmoGrid is a numerical study on the formation of large scale structure in the universe. The simulations are challenging due to the enormous dynamic range in spatial and temporal coordinates, as well as the enormous computer resources required. In CosmoGrid we dealt with the computational requirements by connecting up to four supercomputers via an optical network and make them operate as a single machine. This was challenging, if only for the fact that the supercomputers of our choice are separated by half the planet, as three of them are located scattered across Europe and fourth one is in Tokyo. The co-scheduling of multiple computers and the 'gridification' of the code enabled us to achieve an efficiency of up to $93\%$ for this distributed intercontinental supercomputer. In this work, we find that high-performance computing on a grid can be done much more effectively if the sites involved are willing to be flexible about their user policies, and that having facilities to provide such flexibility could be key to strengthening the position of the HPC community in an increasingly Cloud-dominated computing landscape. Given that smaller computer clusters owned by research groups or university departments usually have flexible user policies, we argue that it could be easier to instead realize distributed supercomputing by combining tens, hundreds or even thousands of these resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
