In recent years, with the slowing down of Moore's law, utilization of hardware other than CPU such as GPU or FPGA is increasing. However, when using heterogeneous hardware other than CPUs, barriers of technical skills such as CUDA and HDL are high. Based on that, I have proposed environment adaptive software that enables automatic conversion, configuration, and high-performance operation of once written code, according to the hardware to be placed. Partly of the offloading to the GPU and FPGA was automated previously. In this paper, I improve and propose a previous automatic GPU offloading method to expand applicapable software and enhance performances more. I evaluate the effectiveness of the proposed method in multiple applications.
In the recent years, systems using FPGAs, GPUs have increased due to their advantages such as power efficiency compared to CPUs. However, use in systems such as FPGAs and GPUs requires understanding hardware-specific technical specifications such as HDL and CUDA, which is a high hurdle. Based on this background, I previously proposed environment adaptive software that enables automatic conversion, configuration, and high-performance operation of once written code according to the hardware to be placed. As an element of the concept, I proposed a method to automatically offload loop statements o…
In recent years, utilization of heterogeneous hardware other than small core CPU such as GPU, FPGA or many core CPU is increasing. However, when using heterogeneous hardware, barriers of technical skills such as CUDA are high. Based on that, I have proposed environment-adaptive software that enables automatic conversion, configuration, and high performance operation of once written code, according to the hardware to be placed. However, the source language for offloading was mainly C/C++ language applications currently, and there was no research for common offloading for various language applic…
In recent years, with the prediction of Moore's law slowing down, utilization of hardware other than CPU such as FPGA which is energy effective is increasing. However, when using heterogeneous hardware other than CPUs, barriers of technical skills such as OpenCL are high. Based on that, I have proposed environment adaptive software that enables automatic conversion, configuration, and high-performance operation of once written code, according to the hardware to be placed. Partly of the offloading to the GPU was automated previously. In this paper, I propose and evaluate an automatic extraction…
In recent years, with the prediction of Moore's law slowing down, utilization of hardware other than CPU such as FPGA which is energy effective is increasing. However, when using heterogeneous hardware other than CPUs, barriers of technical skills such as OpenCL are high. Based on that, I have proposed environment adaptive software that enables automatic conversion, configuration, and high-performance operation of once written code, according to the hardware to be placed. Partly of the offloading to the GPU was automated previously. In this paper, I propose and evaluate an automatic extraction…
For Open IoT, we have proposed Tacit Computing technology to discover the devices that have data users need on demand and use them dynamically and an automatic GPU offloading technology as an elementary technology of Tacit Computing. However, it can improve limited applications because it only optimizes parallelizable loop statements extraction. Thus, in this paper, to improve performances of more applications automatically, we propose an improved method with reduction of data transfer between CPU and GPU. We evaluate our proposed offloading method by applying it to Darknet and find that it ca…
IoT technologies have been progressed. Now Open IoT concept has attracted attentions which achieve various IoT services by integrating horizontal separated devices and services. For Open IoT era, we have proposed the Tacit Computing technology to discover the devices with necessary data for users on demand and use them dynamically. However, existing Tacit Computing does not care about performance and operation cost. Therefore in this paper, we propose an automatic GPU offloading technology as an elementary technology of Tacit Computing which uses Genetic Algorithm to extract appropriate offloa…
Evaluating OpenMP Offloading for Intra-node Multi-GPU Programming across NVIDIA, AMD, and Intel Architectures: A 3D Heat Transfer Case Study
Rudolfovo-Science and Technology Centre Novo mesto（鲁多尔沃夫诺沃梅斯托科学与技术中心）；Faculty of Mechanical Engineering, University of Ljubljana（卢布尔雅那大学机械工程学院）；Department of Computer Science, FSTM, University of Luxembourg（卢森堡大学科学技术学院计算机科学系）
In recent years, utilization of heterogeneous hardware other than small core CPU such as GPU, FPGA or many core CPU is increasing. However, when using heterogeneous hardware, barriers of technical skills such as OpenMP, CUDA and OpenCL are high. Based on that, I have proposed environment-adaptive software that enables automatic conversion, configuration, and high performance operation of once written code, according to the hardware to be placed. However, including existing technologies, there has been no research to properly and automatically offload the mixed offloading destination environmen…
In recent years, utilization of heterogeneous hardware other than small core CPU such as GPU, FPGA or many core CPU is increasing. However, when using heterogeneous hardware, barriers of technical skills such as CUDA are high. Based on that, I have proposed environment-adaptive software that enables automatic conversion, configuration, and high performance and low power operation of once written code, according to the hardware to be placed. I also have verified performance improvement of automatic GPU and FPGA offloading so far. In this paper, I verify low power operation with environment adap…
When using heterogeneous hardware other than CPUs, barriers of technical skills such as OpenCL are high. Based on that, I have proposed environment adaptive software that enables automatic conversion, configuration, and high-performance operation of once written code, according to the hardware to be placed. Partly of the offloading to the GPU was automated previously. In this paper, I propose and evaluate an automatic extraction method of appropriate offload target loop statements of source code as the first step of offloading to FPGA. I evaluate the effectiveness of the proposed method in mul…
ZenFlow: Enabling Stall-Free Offloading for LLM Training
University of Virginia（弗吉尼亚大学）；University of California, Merced（加州大学梅尔塞德斯分校）；Argonne National Laboratory（阿贡国家实验室）；Snowflake AI Research（Snowflake人工智能研究）
TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
SKLP, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China（SKLP，计算技术研究所，中国科学院，北京）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京）
DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs
Institute of Automation, CAS（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Shanghai Jiao Tong University（上海交通大学）；AiRiA
In recent years, utilization of heterogeneous hardware other than small core CPU such as GPU, FPGA or many core CPU is increasing. However, when using heterogeneous hardware, barriers of technical skills such as OpenMP, CUDA and OpenCL are high. Based on that, I have proposed environment-adaptive software that enables automatic conversion, configuration, and high performance operation of once written code, according to the hardware to be placed. However, although the conversion of the code according to the migration destination environment has been studied so far, there has been no research to…
The PLUTO Code on GPUs: Offloading Lagrangian Particle Methods
UNITO - Università degli Studi di Torino（都灵大学）；INAF - Istituto Nazionale di Astrofisica, Osservatorio Astrofisico di Torino（意大利国家天体物理研究所都灵天文台）；ICSC - Italian Research Center on High Performance Computing, Big Data and Quantum Computing（意大利高性能计算、大数据和量子计算研究中心）；CINECA
EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?
The Hong Kong University of Science and Technology（香港科技大学）；Huawei Technologies Ltd.（华为技术有限公司）；The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）
ProMoE: Fast MoE-based LLM Serving using Proactive Caching
Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University（上海交通大学平行与分布式系统研究所）；Key Laboratory of System Software (Chinese Academy of Sciences)（中国科学院系统软件重点实验室）；Zhejiang University（浙江大学）
NaNet is an FPGA-based PCIe X8 Gen2 NIC supporting 1/10 GbE links and the custom 34 Gbps APElink channel. The design has GPUDirect RDMA capabilities and features a network stack protocol offloading module, making it suitable for building low-latency, real-time GPU-based computing systems. We provide a detailed description of the NaNet hardware modular architecture. Benchmarks for latency and bandwidth for GbE and APElink channels are presented, followed by a performance analysis on the case study of the GPU-based low level trigger for the RICH detector in the NA62 CERN experiment, using either…
Designing and implementing efficient, provably correct parallel neural network processing is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. However, the diversity and large-scale data size have posed a significant challenge to construct a flexible and high-performance implementation of deep learning neural networks. To improve the performance and maintain the scalability, we present CNNLab, a novel deep learning framework using GPU and…
