학부 유민수 교수님 연구실에서 ‘인공지능(AI)기반 추천 기술’을 가속화하는 시스템 개발에 성공하여 언론에 보도되었습니다.
이는 AI 기반 추천 알고리즘을 6배에서 최대 17배까지 가속화하는 시스템에 해당합니다.
AI 추천서비스란 흔히 구글이나 네이버 등 포탈 사이트에서 확인할 수 있는 광고 추천 등을 의미합니다. 이는 딥러닝 시스템을 기반으로 하여, AI기술을 통해서 사용자의 검색 기록 등을 활용하여 개개인에 맞는 정보를 추천하여 제공하는 서비스를 의미합니다. 해당 서비스의 핵심은 바로 “알고리즘의 수행 시간”입니다. 이는 실시간 정보를 기반으로 하기에, 얼마나 결과를 재빠르게 도출하느냐가 해당 서비스의 질을 결정합니다. 또한, 이는 나아가 사용자의 만족도 및 기업의 이윤 창출과도 직접적인 연관을 보인다고 할 수 있습니다.
유민수 교수님 연구팀은 ‘메모리를 중심으로 한 AI 가속기 컴퓨팅시스템’을 고안하여 수행시간을 효과적으로 줄일 수 있는 방안을 고안해냈습니다. 바로 흔히 말하는 ‘메모리 병목현상’을 개선한 것입니다. 연구팀에서 제안한 시스템은 메모리 근처에 AI 가속기를 두는 ‘Processing in Memory(PI614M) 기술입니다. 이는 데이터 전송량 및 메로리 엑세스 횟수를 효과적으로 줄였다는 점에서 주목할 만 합니다.
이번에 개발된 기술은 다양한 분야에 활용될 수 있어, 긍정적인 평가를 받고 있습니다. 유민수 교수님께서는 이번에 개발한 기술을 통해서 국내 기업과 협업하여 대한민국이 AI 가속기 시장의 주도권을 따낼 수 있도록 노력하겠다는 입장을 밝혔습니다.
이번 연구성과는 2019년 컴퓨터 시스템 구조 분야에서 발표된 수백편의 논문들 중에서 가장 임팩트 있는 연구 결과 26편에 수여된 2019 IEEE Micro Top Picks — Honorable Mention 리스트에 등재되는 등 그 연구의 우수성을 세계적으로 인정받았습니다.
한편 이번 연구는 유민수 교수님 연구팀이 삼성전자 미래기술 육성재단으로부터 지원을 받아 수행되었습니다.
Title: TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning
Authors: Youngeun Kwon, Yunjae Lee, and Minsoo Rhu
Recent studies from several hyper scalars pinpoint to embedding layers as the most memory-intensive deep learning (DL) algorithm being deployed in today’s data centers. This paper addresses the memory capacity and bandwidth challenges of embedding layers and the associated tensor operations. We present our vertically integrated hardware/software co-design, which includes a custom DIMM module enhanced with near-memory processing cores tailored for DL tensor operations. These custom DIMMs are populated inside a GPU-centric system interconnect as a remote memory pool, allowing GPUs to utilize for scalable memory bandwidth and capacity expansion. A prototype implementation of our proposal on real DL systems shows an average 6.2−17.6× performance improvement on state-of-the-art DNN-based recommender systems.
우리 학부 심현철 교수 연구팀이 지난 7월 12일(금) 과학기술정보통신부가 주최하고 정보통신기술평가원이 주관한 `2019 인공지능 연구개발 그랜드챌린지’ 대회의 제어지능 분야에 출전해 1위를 차지했습니다.
지난 2017년 처음 개최된 `인공지능 연구개발 그랜드챌린지’는 높은 경제‧사회적 파급 효과가 기대되지만, 기술 난이도가 높아 해결되지 않은 문제를 발굴하고 인공지능 기술을 활용해 이를 해결하기 위한 대회입니다.
올해 열린 2회 대회는 드론을 활용하여 다양한 복합재난에 대응할 수 있는 기본 인지지능 소프트웨어 기술 개발이 도전 과제로 제시되었으며 오는 2022년까지 4년에 걸쳐 소프트웨어와 하드웨어를 융합해 문제를 해결하고 최종 목표를 달성하는 `그랜드 챌린지’ 형태로 확대되었습니다.
△시각지능(상황인지), △시각지능(문자인지), △청각지능(음향인지), △제어지능(로보틱스 제어) 등 총 4개 트랙으로 구성되어 각 분야별로 상위 3개 팀이 내년에 열리는 2단계 대회 출전권을 얻게 됩니다.
심 교수 연구팀이 출전한 제어지능 분야는 제시된 사양으로 제작된 드론을 활용해 미로·창문·나무·기둥·테니스 네트·강풍 등의 조합으로 구성된 장애물 세트를 완전 자율비행으로 통과하는 종목입니다.
인터넷 연결 없이 드론에 탑재된 인공지능만으로 미션을 수행해야 하는 고난이도 과제로 8개의 출전팀 중 대부분의 팀이 3구간을 통과하지 못해 탈락했습니다. 심 교수팀은 유일하게 4구간인 파이프 통과 미션을 성공하는 드론 제어 기술을 선보여 1위를 차지했습니다.
우승을 차지한 심현철 교수 연구팀은 과학기술부 장관상을 받았으며 상금 3백만 원과 향후 연구비로 총 6억 원을 지원받게 됩니다. 트랙별로 입상한 상위 3개 팀은 대회 종료 후 후속 연구를 지속해 2020년 12월에 예정된 2단계 대회에서 연구 성과를 평가받게 됩니다.
한편, 이번 열린 1단계 대회에는 고등학생부터 대학생·기업·연구소 등의 다양한 배경의 참가자들이 출전했으며 18세 최연소 학생부터 최고령 68세 참가자까지 총 121개 팀 617명이 경쟁을 펼쳤습니다.
The research study authored by Daewoo Kim, Sangwoo Moon, David Hostallero, Wan Ju Kang, Taeyoung Lee, Kyunghwan Son, and Yung Yi was accepted at 7th International Conference on Learning Representations (ICLR 2019)
Title: Learning to Schedule Communication in Multi-agent Reinforcement Learning
Authors: Daewoo Kim, Sangwoo Moon, David Hostallero, Wan Ju Kang, Taeyoung Lee, Kyunghwan Son, and Yung Yi
We present a first-of-its-kind study of communication-aided cooperative multi-agent reinforcement learning tasks with two realistic constraints on communication: (i) limited bandwidth and (ii) medium contention. Bandwidth limitation restricts the amount of information that can be exchanged by any agent accessing the medium for inter-agent communication, whereas medium contention confines the number of agents to access the channel itself, as in the state-of-the-art wireless networking standards, such as 802.11. These constraints call for a certain form of scheduling. In that regard, we propose a multi-agent deep reinforcement learning framework, called SchedNet, in which agents learn how to schedule themselves, how to encode messages, and how to select actions based on received messages and on its own observation of the environment. SchedNet enables the agents to decide, in a distributed manner, among themselves who should be entitled to broadcasting their encoded messages, by learning to gauge the importance of their (partially) observed information. We evaluate SchedNet against multiple baselines under two different applications, namely, cooperative communication and navigation, and predator-prey. Our experiments show a non-negligible performance gap between SchedNet and other mechanisms such as the ones without communication with vanilla scheduling methods, e.g., round robin, ranging from 32% to 43%.
The research study authored by Joon-Ki Hong* (KAIST EE), Soon-Il Kwon* (SNUH), Eue-Keun Choi (SNUH), Euijae Lee (SNUH), David Earl Hostallero (KAIST EE), Wan-Ju Kang (KAIST EE), Byunghwan Lee (Skylabs), Eui-Rim Jeong (Hanbat University), Bon-Kwon Koo (SNUH), Seil Oh (SNUH), Yung Yi (KAIST EE) was accepted at JMIR mHealth and uHealth 7.6(2019)
Title: Deep Learning Approaches to Detect Atrial Fibrillation Using Photoplethysmographic Signals: Algorithms Development Study
Authors: Joonki Hong*, Soonil Kwon*, Eue-Keun Choi, Euijae Lee, David Earl Hostallero, Wan Ju Kang, Byunghwan Lee, Eui-Rim Jeong, Bon-Kwon Koo, Seil Oh, Yung Yi (* these authors contributed equally)
Wearable devices have evolved as screening tools for atrial fibrillation (AF). A photoplethysmographic (PPG) AF detection algorithm was developed and applied to a convenient smartphone-based device with good accuracy. However, patients with paroxysmal AF frequently exhibit premature atrial complexes (PACs), which result in poor unmanned AF detection, mainly because of rule-based or handcrafted machine learning techniques that are limited in terms of diagnostic accuracy and reliability. We developed deep learning (DL) based AF classifiers based on 1-dimensional convolutional neural network (1D-CNN) and recurrent neural network (RNN) architectures and examined 75 patients with AF who underwent successful elective direct-current cardioversion (DCC). New DL classifiers could detect AF using PPG monitoring signals with high diagnostic accuracy (97.58 %) even with frequent PACs and could outperform previously developed AF detectors. Although diagnostic performance decreased as the burden of PACs increased, performance improved when samples from more patients were trained. Moreover, the reliability of the diagnosis could be indicated by the confidence level (CL). Wearable devices sensing PPG signals with DL classifiers should be validated as tools to screen for AF.
Title: Beyond the Memory Wall: A Case for Memory-centric HPC System for Deep Learning
Authors: Youngeun Kwon and Minsoo Rhu
As the models and the datasets to train deep learning (DL) models scale, computer system architects are faced with new challenges, one of which is the memory capacity bottleneck, where the limited physical memory inside the accelerator device constrains the algorithm that can be studied. We propose a memory-centric deep learning system (MC-DLA) that can transparently expand the memory capacity available to the accelerators while also providing fast inter-device communication for parallel training. Our proposal aggregates a pool of memory modules locally within the device-side interconnect, which are decoupled from the host interface and function as a vehicle for transparent memory capacity expansion. Compared to device-centric DL systems (DC-DLA) such as NVIDIA’s DGX, our proposal achieves an average 2.8x speedup on eight DL applications and increases the system-wide memory capacity to tens of TBs.
Authors: Sejun Park, Eunho Yang, Se-Young Yun, Jinwoo Shin
Graphical models (GMs) have been successfully applied to various applications of machine learning. Given a GM, computing its partition function is the most essential inference task, but it is computationally intractable in general. To address the issue, iterative approximation algorithms exploring certain local structure/consistency of GM have been investigated as popular choices in practice. However, due to their local/iterative nature, they often output poor approximations or even do not converge, e.g., in low-temperature regimes (hard instances of large parameters). To overcome the limitation, we propose a novel approach utilizing the global spectral feature of GM. Our contribution is two-fold: (a) we first propose a fully polynomial-time approximation scheme (FPTAS) for approximating the partition function of GM associating with a low-rank coupling matrix; (b) for general high-rank GMs, we design a spectral mean-field scheme utilizing (a) as a subroutine, where it approximates a high-rank GM into a product of rank-1 GMs for an efficient approximation of the partition function. The proposed algorithm is more robust in its running time and accuracy than prior methods, i.e., neither suffers from the convergence issue nor depends on hard local structures. Our experiments demonstrate that it indeed outperforms baselines, in particular, significantly in the low-temperature regimes.
Figure1. An illustration of the spectral approximate inference for the partition function approximation
Title: Using Pre-Training Can Improve Model Robustness and Uncertainty
Authors: Dan Hendrycks (UC Berkeley), Kimin Lee (KAIST EE), Mantas Mazeika (University of Chicago)
He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training. We show that although pre-training may not improve performance on traditional classification metrics, it improves model robustness and uncertainty estimates. Through extensive experiments on label corruption, class imbalance, adversarial examples, out-of-distribution detection, and confidence calibration, we demonstrate large gains from pre-training and complementary effects with task-specific methods. We show approximately a 10% absolute improvement over the previous state-of-the-art in adversarial robustness. In some cases, using pre-training without task-specific methods also surpasses the stateof-the-art, highlighting the need for pre-training when evaluating future methods on robustness and uncertainty tasks.
Figure 1. Training for longer is not a suitable strategy for label corruption. By training for longer, the network eventually begins to model and memorize label noise, which harms its overall performance. Labels are corrupted uniformly to incorrect classes with 60% probability, and the Wide Residual Network classifier has learning rate drops at epochs 80, 120, and 160.
Title: Robust Inference via Generative Classifiers for Handling Noisy Labels
Authors: Ki-Min Lee (KAIST EE), Suk-Min Yun (KAIST EE), Ki-Bok Lee (CS UMich), Hong-Lak Lee (CS Umich / Google Brain), Bo Li (CS UIUC) and Jin-Woo Shin (KAIST EE)
Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets. To mitigate the issue, we propose a novel inference method, termed Robust Generative classifier (RoG), applicable to any discriminative (e.g., softmax) neural classifier pre-trained on noisy datasets. In particular, we induce a generative classifier on top of hidden feature spaces of the pre-trained DNNs, for obtaining a more robust decision boundary. By estimating the parameters of generative classifier using the minimum covariance determinant estimator, we significantly improve the classification accuracy with neither re-training of the deep model nor changing its architectures. With the assumption of Gaussian distribution for features, we prove that RoG generalizes better than baselines under noisy labels. Finally, we propose the ensemble version of RoG to improve its performance by investigating the layer-wise characteristics of DNNs. Our extensive experimental results demonstrate the superiority of RoG given different learning models optimized by several training techniques to handle diverse scenarios of noisy labels.
Figure 1. Visualization of features on the penultimate layer using t-SNE from training samples when the noise fraction is 20%
_200706_1.png "AI in Computer division 1")
