Temporal Graph Neural Networks (TGNNs) are powerful models to capture temporal, structural, and contextual information on temporal graphs. The generated temporal node embeddings outperform other methods in many downstream tasks. Real-world applications require high performance inference on real-time streaming dynamic graphs. However, these models usually rely on complex attention mechanisms to capture relationships between temporal neighbors. In addition, maintaining vertex memory suffers from intrinsic temporal data dependency that hinders task-level parallelism, making it inefficient on general-purpose processors. In this work, we present a novel model-architecture co-design for inference in memory-based TGNNs on FPGAs. The key modeling optimizations we propose include a light-weight method to compute attention scores and a related temporal neighbor pruning strategy to further reduce computation and memory accesses. These are holistically coupled with key hardware optimizations that leverage FPGA hardware. We replace the temporal sampler with an on-chip FIFO based hardware sampler and the time encoder with a look-up-table. We train our simplified models using knowledge distillation to ensure similar accuracy vis-á-vis the original model. Taking advantage of the model optimizations, we propose a principled hardware architecture using batching, pipelining, and prefetching techniques to further improve the performance. We also propose a hardware mechanism to ensure the chronological vertex updating without sacrificing the computation parallelism. We evaluate the performance of the proposed hardware accelerator on three real-world datasets.
Low Latency Edge Classification GNN for Particle Trajectory Tracking on FPGAs
National Yang Ming Chiao Tung University(国立阳明交通大学);University of California San Diego(加利福尼亚大学圣迭戈分校);University of Washington(华盛顿大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
PowerGear: Early-Stage Power Estimation in FPGA HLS via Heterogeneous Edge-Centric GNNs
Peng Cheng Laboratory(鹏城实验室);The University of Auckland(奥克兰大学);Shanghai Jiao Tong University(上海交通大学);The Hong Kong University of Science and Technology(香港科技大学);Peking University(北京大学)
LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics
Imperial College London(伦敦帝国理工学院);Southeast University(东南大学);AMD Adaptive and Embedded Computing Group (AECG) Labs(AMD自适应与嵌入式计算集团实验室);European Organization for Nuclear Research (CERN)(欧洲核子研究中心)
Graph Neural Networks for Charged Particle Tracking on FPGAs
University of Pennsylvania(宾夕法尼亚大学);University of California San Diego(加州大学圣地亚哥分校);National Yang Ming Chiao Tung University(国立阳明交通大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Princeton University(普林斯顿大学);University of Washington(华盛顿大学)
End-to-End Keyword Spotting on FPGA Using Graph Neural Networks with a Neuromorphic Auditory Sensor
AGH University of Krakow, Poland(克拉科夫AGH大学,波兰);Embedded Vision Systems Group(嵌入式视觉系统组);Computer Vision Laboratory(计算机视觉实验室);Robotics and Technology of Computers Lab.(机器人与计算机技术实验室);ETSII, EPS, SCORE, I3US, Universidad de Sevilla(塞维利亚大学ETSII、EPS、SCORE、I3US)
The Logical Expressiveness of Temporal GNNs via Two-Dimensional Product Logics
RPTU Kaiserslautern-Landau(亚琛工业大学凯撒斯劳滕-兰道分校);Queen Mary University of London(伦敦女王大学);University of Łódź(卢宾大学);Theoretical Computer Science / Formal Methods(理论计算机科学/形式方法);University of Kassel(卡塞尔大学)
KAT-GNN: A Knowledge-Augmented Temporal Graph Neural Network for Risk Prediction in Electronic Health Records
Institute of Computer Science and Engineering, National Yang Ming Chiao Tung University(计算机科学与工程研究所,国立阳明交通大学);School of Medicine, National Tsing Hua University(医学院,国立清华大学);Department of Computer Science, National Yang Ming Chiao Tung University(计算机科学系,国立阳明交通大学);Computational Health Informatics Program, Boston Children’s Hospital(计算健康信息学计划,波士顿儿童医院)
Geometric GNNs for Charged Particle Tracking at GlueX
Thomas Jefferson National Accelerator Facility(托马斯·杰斐逊国家加速器设施);Department of Computer Science, University of Houston(休斯顿大学计算机科学系);Department of Physics, William & Mary(威廉与玛丽学院物理系);Computer Science Department, Old Dominion University(老 Dominion 大学计算机科学系)
TRINE: A Token-Aware, Runtime-Adaptive FPGA Inference Engine for Multimodal AI
University of California, Irvine(加州大学尔湾分校);Purdue University Northwest(北达科他州立大学);Qualcomm(高通);Samsung(三星)
针对多模态AI中不同计算/内存模式导致嵌入式平台实时性不足的问题,提出TRINE,一种无需重配置的单比特流FPGA加速器与编译器,通过统一层映射、运行时模式切换、令牌剪枝和依赖感知层卸载,实现端到端多模态推理,在Alveo U50和ZCU104上相比RTX 4090和Jetson Orin Nano分别降低延迟22.57倍和6.86倍,功耗仅20-21W。
Never Skip a Batch: Dense Learning of Temporal GNNs via Adaptive Pseudo-Supervision
Risk AI Research Lab(风险人工智能研究实验室);Moscow Independent Research Institute of Artificial Intelligence(莫斯科独立人工智能研究所);Applied AI Institute(应用人工智能研究所);Beijing Institute of Mathematical Sciences and Applications(北京雁栖湖应用数学研究院)
TFE-GNN: A Temporal Fusion Encoder Using Graph Neural Networks for Fine-grained Encrypted Traffic Classification
Tsinghua University(清华大学);The Hong Kong Polytechnic University(香港理工大学);Peng Cheng Laboratory(鹏城实验室);University of Molise(莫利塞大学);IIT-CNR(意大利国家研究委员会信息与电信研究所);Chinese Academy of Sciences(中国科学院)
A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learning
North China Electric Power University(华北电力大学);Beijing University of Posts and Telecommunications(北京邮电大学);Beihang University(北京航空航天大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)
When GNNs Fail: Quantifying and Overcoming Temporal Correlation Volatility in Time Series
Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);The Hong Kong University of Science and Technology(香港科技大学);East China Normal University(华东师范大学);University of Michigan(密歇根大学)
Towards Expressive Spectral-Temporal Graph Neural Networks for Time Series Forecasting
Griffith University(格里菲斯大学);Monash University(莫纳什大学);University of Stuttgart(斯图加特大学);Alibaba Group(阿里巴巴集团);University of New South Wales(新南威尔士大学);Emory University(埃默里大学);Squirrel Ai Learning