AI in EE

AI IN DIVISIONS

AI in Computer Division

AI in EE

AI IN DIVISIONS

AI in Computer Division ​

AI in Computer Division

Accelerating GNN Training with Locality-Aware Partial Execution (Awarded Best Paper)

Taehyun Kim, Changho Hwang, Kyoungsoo Park, Zhiqi Lin, Peng Cheng, Youshan Miao, Lingxiao Ma, and Yongqiang Xiong

In Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys ’21)

August 2021

 

 

Abstract

Graph Neural Networks (GNNs) are increasingly popular for various prediction and recommendation tasks. Unfortunately, the graph datasets for practical GNN applications are often too large to fit into the memory of a single GPU, leading to frequent data loading from host memory to GPU. This data transfer overhead is highly detrimental to the performance, severely limiting the training throughput. In this paper, we propose locality-aware, partial code execution that significantly cuts down the data copy overhead for GNN training. The key idea is to exploit the “near-data” processors for the first few operations in each iteration, which reduces the data size for DMA operations. In addition, we employ task scheduling tailored to GNN training and apply load balancing between CPU and GPU. We find that our approach substantially improves the performance, achieving up to 6.6x speedup in training throughput over the state-of-the-art system design.

 

Copyright ⓒ 2015 KAIST Electrical Engineering. All rights reserved. Made by PRESSCAT

Copyright ⓒ 2015 KAIST Electrical Engineering. All rights reserved. Made by PRESSCAT

34141 대전광역시 유성구 대학로 291
한국과학기술원(KAIST)
Tel. 042-350-3411   Fax. 042-350-3410

Copyright ⓒ 2015 KAIST Electrical
Engineering. All rights reserved.

Made by PRESSCAT