Archives: AI in Computer division

EE Professor Rhu, Minsoo’s Research Team Build First-ever Privacy-aware Artificial Intelligence Semiconductor, Speeding Up the Differentially Private Learning Process 3.6 Times Google’s TPUv3

 

[Professor Rhu, Minsoo]

EE Professor Rhu and his research team have taken artificial intelligence semiconductors a big leap forward in the application of differentially private machine learning. Professor Rhu’s team analyzed the bottleneck component in the differentially private machine learning performance and devised a semiconductor chip greatly improving differentially private machine learning application performance.

Professor Rhu’s artificial intelligence chip consists of, among others, a cross-product-based arithmetic unit and an addition tree-based post-processing arithmetic unit and is capable of 3.6 times faster machine learning process compared with that of Google’s TPUv3, today’s most widely used AI processor.

The new chip also boasts comparable performance to that of NVIDIA’s A100 GPU, even with 10 times less resources.

[From left, Co-lead authors Park, Beomsik and Hwang, Ranggi; co-authors Yoon, Dongho and Choi, Yoonhyuk]

This work, with EE researchers Park, Beomsik and Hwang, Ranggi as co-first authors, will be presented as DiVa: An Accelerator for Differentially Private Machine Learning at the 55th IEEE/ACM International Symposium on Microarchitectures (MICRO 2022), the premier research venue for computer architecture research coming October 1 through 5 in Chicago, USA.

Professor Rhu’s achievements have been reported in multiple press coverage.

Links:

AI Times: http://www.aitimes.com/news/articleView.html?idxno=146435

Yonhap : https://www.yna.co.kr/view/AKR20201116072400063?input=1195m

Financial News : https://www.fnnews.com/news/202208212349474072

Donga Science : https://www.dongascience.com/news.php?idx=55893

Industry News : http://www.industrynews.co.kr/news/articleView.html?idxno=46829

[Professor Song Min Kim and first author Kang Min Bae, from left to right]

On the 28th, School of EE professor Song Min Kim’s research team has announced that they have succeeded in creating the world’s first mmWave backscatter system for massive IoT connectivity.

The research, (OmniScatter: extreme sensitivity mmWave backscattering using commodity FMCW radar), led by Kang Min Bae as first author, was presented at ACM MobiSys 2022 this June, and was presented with the best paper award. This is meaningful as it marks the second consecutive year in which the best paper award was presented to a paper belonging to a research group at KAIST’s School of Electrical Engineering.

The backscatter technology described by this research team can greatly reduce the maintenance cost as it operates on ultra-lower power of less than 10 μW, being able to run on a single battery for more than 40 years.

By enabling connectivity on a scale that far exceeds the network density required by next gen communication technologies such as 5G and 6G, this system may serve as a great potential for serving as a stepping stone for the upcoming hyperconnected era.

“mmWave backscattering is a dreamlike technology that can run IoT devices on a large scale, which can drive massive communications at ultra-low power compared to any other technology,” said Professor Song Min Kim. “We hope that this technology will be actively used for the upcoming era of Internet of Things,” he added.

The research was made possible by the funding from Samsung Future Technology Development Project and the Institute for Information & Communication Technology Planning & Evalution.

[Fig 1. Tags used for massive IoT communications (as depicted by red triangles). Over 1100 tags are able to communicate simultaneously without any conflicts]

[Prof. Sung-Ju Lee, Jaemin Shin (KAIST PhD Candidate),  Prof. Yunxin Liu of  Tsinghua University , Prof. Yuanchun Li  of  Tsinghua University, from left]

The research team led by Prof. Sung-Ju Lee of KAIST has published a paper “FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients” at ACM MobiSys (International Conference on Mobile Systems, Applications, and Services) 2022. Founded in 2003, MobiSys has been a premier conference on Mobile Computing and Systems. This year, 38 out of 176 submitted papers have been accepted to be presented at the conference. 

Jaemin Shin (KAIST PhD Candidate) was the lead author, and this work was in collaboration with Tsinghua University in China (Professors Yuanchun Li and Yunxin Liu participated).

Federated Learning is a recent machine learning paradigm proposed by Google that trains on a large corpus of private user data without collecting them. The authors developed a systematic federated learning framework that accelerates the global learning process of federated learning. The new framework actively measures the contribution of each training sample of clients and selects the optimal samples to optimize the training process. The authors also included an adaptive deadline control scheme with varying training data, and achieved 4.5 times speedup in global learning process without sacrificing the model accuracy.

Prof. Lee stated that “Federated learning is an important technology used by many top companies. With the accelerated training time achieved by this research, it has become even more attractive for real deployments. Moreover, our technology has shown to work well in different domains such as computer vision, natural language processing, and human activity recognition.”

Abstract

Fairness and robustness are critical elements of Trustworthy AI that need to be addressed together. Fairness is about learning an unbiased model while robustness is about learning from corrupted data, and it is known that addressing only one of them may have an adverse affect on the other. In this work, we propose a sample selection-based algorithm for fair and robust training. To this end, we formulate a combinatorial optimization problem for the unbiased selection of samples in the presence of data corruption. Observing that solving this optimization problem is strongly NP-hard, we propose a greedy algorithm that is efficient and effective in practice. Experiments show that our algorithm obtains fairness and robustness that are better than or comparable to the state-of-the-art technique, both on synthetic and benchmark real datasets. Moreover, unlike other fair and robust training baselines, our algorithm can be used by only modifying the sampling step in batch selection without changing the training algorithm or leveraging additional clean data.

abstract

Responsible AI is becoming critical as AI is widely used in our everyday lives. Many companies that deploy AI publicly state that when training a model, we not only need to improve its accuracy, but also need to guarantee that the model does not discriminate against users (fairness), is resilient to noisy or poisoned data (robustness), is explainable, and more. In addition, these objectives are not only relevant to model training, but to all steps of end-to-end machine learning, which include data collection, data cleaning and validation, model training, model evaluation, and model management and serving. Finally, responsible AI is conceptually challenging, and supporting all the objectives must be as easy as possible. We thus propose three key research directions towards this vision – depth, breadth, and usability – to measure progress and introduce our ongoing research. First, responsible AI must be deeply supported where multiple objectives like fairness and robust must be handled together. To this end, we propose FR-Train, a holistic framework for fair and robust model training in the presence of data bias and poisoning. Second, responsible AI must be broadly supported, preferably in all steps of machine learning. Currently we focus on the data pre-processing steps and propose Slice Tuner, a selective data acquisition framework for training fair and accurate models, and MLClean, a data cleaning framework that also improves fairness and robustness. Finally, responsible AI must be usable where the techniques must be easy to deploy and actionable. We propose FairBatch, a batch selection approach for fairness that is effective and simple to use, and Slice Finder, a model evaluation tool that automatically finds problematic slices. We believe we scratched the surface of responsible AI for end-to-end machine learning and suggest research challenges moving forward.

Abstract

The deep learning-based satellite image analysis and retraining systems are getting emerging technologies to enhance the capability of the sophisticated analysis of terrestrial objects. In principle, to apply the explainable DNN model for the process of satellite image analysis and retraining, we consider a new acceleration scheduling mechanism. Especially, the conventional DNN acceleration schemes cause serious performance degradation due to computational complexity and costs in satellite image analysis and retraining. In this article, to overcome the performance degradation, we propose cooperative scheduling schemes for explainable DNN acceleration in analysis and retraining process. For the purpose of it, we define the latency and energy cost modeling to derive the optimized processing time and cost required for explainable DNN acceleration. Especially, we show a minimum processing cost considered in the proposed scheduling via layer-level management of the explainable DNN on FPGA-GPU acceleration system. In addition, we evaluate the performance using an adaptive unlabeled data selection scheme with confidence threshold and a semi-supervised learning driven data parallelism scheme in accelerating retraining process. The experimental results demonstrate that the proposed schemes reduce the energy cost of the conventional DNN acceleration systems by up to about 40% while guaranteeing the latency constraints.

Changho Hwang, Taehyun Kim, Sunghyun Kim, Jinwoo Shin, and Kyoungsoo Park

In Proceedings of the 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’21)

April 2021

Abstract

Resource allocation and scheduling strategies for deep learning training (DLT) jobs have a critical impact on their average job completion time (JCT). Unfortunately, traditional algorithms such as Shortest-Remaining-Time-First (SRTF) often perform poorly for DLT jobs. This is because blindly prioritizing only the short jobs is suboptimal and job-level resource preemption is too coarse-grained for effective mitigation of head-of-line blocking. We investigate the algorithms that accelerate DLT jobs. Our analysis finds that (1) resource efficiency often matters more than short job prioritization and (2) applying greedy algorithms to existing jobs inflates average JCT due to overly optimistic views toward future resource availability. Inspired by these findings, we propose Apathetic Future Share (AFS) that balances resource efficiency and short job prioritization while curbing unrealistic optimism in resource allocation. To bring the algorithmic benefits into practice, we also build CoDDL, a DLT system framework that transparently handles automatic job parallelization and efficiently performs frequent share re-adjustments. Our evaluation shows that AFS outperforms Themis, SRTF, and Tiresias-L in terms of average JCT by up to 2.2x, 2.7x, and 3.1x, respectively.

Taehyun Kim, Changho Hwang, Kyoungsoo Park, Zhiqi Lin, Peng Cheng, Youshan Miao, Lingxiao Ma, and Yongqiang Xiong

In Proceedings of the 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys ’21)

August 2021

Abstract

Graph Neural Networks (GNNs) are increasingly popular for various prediction and recommendation tasks. Unfortunately, the graph datasets for practical GNN applications are often too large to fit into the memory of a single GPU, leading to frequent data loading from host memory to GPU. This data transfer overhead is highly detrimental to the performance, severely limiting the training throughput. In this paper, we propose locality-aware, partial code execution that significantly cuts down the data copy overhead for GNN training. The key idea is to exploit the “near-data” processors for the first few operations in each iteration, which reduces the data size for DMA operations. In addition, we employ task scheduling tailored to GNN training and apply load balancing between CPU and GPU. We find that our approach substantially improves the performance, achieving up to 6.6x speedup in training throughput over the state-of-the-art system design.