Evaluation of a Simple, Scalable, Parallel Best-First Search Strategy
Tokyo Institute of Technology(东京工业大学);The University of Tokyo(东京大学);NICTA(澳大利亚国家信息与通信技术研究院);The Australian National University(澳大利亚国立大学);IBM Research(IBM研究院)
Akihiro Kishimoto, Alex Fukunaga, Adi Botea
备注:
in press, to appear in Artificial Intelligence
发表信息:
Artificial Intelligence (2013), vol. 195, pp. 222-248
Large-scale, parallel clusters composed of commodity processors are increasingly available, enabling the use of vast processing capabilities and distributed RAM to solve hard search problems. We investigate Hash-Distributed A* (HDA*), a simple approach to parallel best-first search that asynchronously distributes and schedules work among processors based on a hash function of the search state. We use this approach to parallelize the A* algorithm in an optimal sequential version of the Fast Downward planner, as well as a 24-puzzle solver. The scaling behavior of HDA* is evaluated experimentally on a shared memory, multicore machine with 8 cores, a cluster of commodity machines using up to 64 cores, and large-scale high-performance clusters, using up to 2400 processors. We show that this approach scales well, allowing the effective utilization of large amounts of distributed memory to optimally solve problems which require terabytes of RAM. We also compare HDA* to Transposition-table Driven Scheduling (TDS), a hash-based parallelization of IDA*, and show that, in planning, HDA* significantly outperforms TDS. A simple hybrid which combines HDA* and TDS to exploit strengths of both algorithms is proposed and evaluated.
TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences(中国科学院大学杭州高等研究院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Science and Technology of China(中国科学技术大学)