Archives: AI in Computer division

Our department’s Professor Myounsoo Jung’s research team has developed the world’s first AI semiconductor for search engines based on CXL 3.0.

 

Approximate nearest neighbor search (ANNS) is widely used in commercial services such as image search, database, and recommendation systems.

However, in production-level ANNS, there is a challenge of requiring a large amount of memory due to the extensive dataset.

To address this memory pressure issue, modern ANNS techniques leverage lossy compression methods or employ persistent storage for their memory expansion.

However, these approaches often suffer from low accuracy and performance.

 

The research team proposed expanding memory capacity via compute express link (CXL), which is PCIe based open-industry interconnect technology that allows the underlying working memory to be highly scalable and composable at a low cost.

Furthermore, the use of a CXL switch enables connecting multiple memory expanders to a single port, providing greater scalability. However, memory expansion through CXL has the disadvantage of increased memory access time compared to local memory.

 

The research team has developed an AI semiconductor, ‘CXL-ANNS‘, which leverages CXL switch and memory expanders to accommodate high memory pressure that comes from extensive datasets without losing accuracy or performance.

Additionally, by using near data processing and data partitioning based on locality, the performance of CXL-ANNS is improved.

They also compared prototyped CXL-ANNS with the existing solutions for ANNS. Compared to previous research, CXL-ANNS shows 111 times higher performance. Particularly, 92 times higher performance can be achieved compared to Microsoft’s solution that is used in commercial service.

 

This research, along with the paper titled “CXL-ANNS: Software-Hardware Collaborative Memory Disaggregation and Computation for Billion-Scale Approximate Nearest Neighbor Search”, will be presented in July at ‘USENIX Annual Technical Conference, ATC, 2023’.

 

 

 

 

The research was supported by Panmnesia (http://panmnesia.com). More information on this paper can be found at CAMELab website (http://camelab.org).

 

[News Link]

The Korea Economic Daily: https://www.hankyung.com/it/article/202305259204i

The Herald Business: http://news.heraldcorp.com/view.php?ud=20230525000225

ChosunBiz: https://biz.chosun.com/science-chosun/technology/2023/05/25/4UW5LPX3WVARVIS3QBBICPINFM/

etnews: https://www.etnews.com/20230525000092

[Prof. David Hyunchul Shim]

 

KAIST PhD candidate Yuji Roh from the School of Electrical Engineering (advisor: Prof. Steven Euijong Whang) was selected as a recipient of the 2022 Microsoft Research PhD Fellowship. 

[Yuji Roh]

 

The list of recipients: https://www.microsoft.com/en-us/research/academic-program/phd-fellowship/2022-recipients/

Interview (Asia): https://www.youtube.com/watch?v=qwq3R1XU8UE

 

 

[Prof. Minsoo Rhu]

 

 

[Prof. Myoungsoo Jung, Miryeong Kwon, Seungjun Lee, and Hyunkyu Cho from left]

 

Our department’s Professor Myoungsoo Jung’s research team has developed the world’s first Predictable Latency Mode (PLM) SSD based hardware and software co-designed framework for Log-Structured Merge Key-Value Stores (LSM KV store).

 

The research team has developed the ‘hardware and software co-designed framework for LSM KV store, Vigil-KV’ that eliminates long-tail latency by utilizing the Predictable Latency Mode (PLM) interface, which provides constant read latency, to the actual datacenter-scale SSD. Vigil-KV outpoerforms 3.19x faster tail latency and 34% faster average latency compared to the existing LSM KV store.

 

LSM KV store, a kind of database, is used to manage various application data, and it must process the user requests within the requirement time in order not to degrade the user experience. To this end, Vigil-KV enables a predictable latency mode (PLM) interface on an actual datacenter-scale NVMe SSD (PLM SSD), which guarantees constant read latency in deterministic mode related to read service without performing SSD’s internal tasks.

 

Specifically, Vigil-KV hardware makes the deterministic mode SSDs exist in the system to remove SSD’s internal tasks by configuring PLM SSD RAID. In addition, Vigil-KV software prevents the deterministic mode from being released by LSM KV store’s internal tasks, scheduling LSM KV store operations (e.x., compaction/flush operations) and client requests.

 

Among the proposed research results, especially noteworthy is that Vigil-KV is the first work that implements the PLM interface in a real SSD and makes the read latency of LSM KV store deterministic in a hardware-software co-design manner. They prototype Vigil-KV hardware on a 1.92TB datacenter-scale NVMe SSD while implementing Vigil-KV software using Linux 4.19.91 and RocksDB 6.23.0.

 

The KAIST Ph.D. Candidates (Miryeong Kwon, Seungjun Lee, and Hyunkyu Choi) participate in this research, and the paper (Vigil-KV: Hardware-Software Co-Design to Integrate Strong Latency Determinism into Log-Structured Merge Key-Value Stores) was reported in July, 11th at ‘USENIX Annual Technical Conference, ATC, 2022’. In addition, they has won the Best Paper Award from Samsung for this paper (Vigil-KV) with Professor Jae-Hyeok Choi’s research team.

 

The Best Paper Award from Samsung recognizes master’s and doctorate students that participated in research grant projects and published papers related to the project among papers adopted by foreign journals/conferences since September 21st. This year’s awards consisted of grand award (2 people), excellence award (1 person), and encouragement award (2 people).

The research was supported by Samsung. More information on this paper can be found at http://camelab.org.

 

 

 

 

 

 

 

EE Professor Rhu, Minsoo’s Research Team Build First-ever Privacy-aware Artificial Intelligence Semiconductor, Speeding Up the Differentially Private Learning Process 3.6 Times Google’s TPUv3

 

[Professor Rhu, Minsoo]

 

EE Professor Rhu and his research team have taken artificial intelligence semiconductors a big leap forward in the application of differentially private machine learning. Professor Rhu’s team analyzed the bottleneck component in the differentially private machine learning performance and devised a semiconductor chip greatly improving differentially private machine learning application performance.

Professor Rhu’s artificial intelligence chip consists of, among others, a cross-product-based arithmetic unit and an addition tree-based post-processing arithmetic unit and is capable of 3.6 times faster machine learning process compared with that of Google’s TPUv3, today’s most widely used AI processor.

The new chip also boasts comparable performance to that of NVIDIA’s A100 GPU, even with 10 times less resources.

 

[From left, Co-lead authors Park, Beomsik and Hwang, Ranggi; co-authors Yoon, Dongho and Choi, Yoonhyuk]

 

This work, with EE researchers Park, Beomsik and Hwang, Ranggi as co-first authors, will be presented as DiVa: An Accelerator for Differentially Private Machine Learning at the 55th IEEE/ACM International Symposium on Microarchitectures (MICRO 2022), the premier research venue for computer architecture research coming October 1 through 5 in Chicago, USA.

Professor Rhu’s achievements have been reported in multiple press coverage.

 

Links:

AI Times: http://www.aitimes.com/news/articleView.html?idxno=146435

Yonhap : https://www.yna.co.kr/view/AKR20201116072400063?input=1195m

Financial News : https://www.fnnews.com/news/202208212349474072

Donga Science : https://www.dongascience.com/news.php?idx=55893

Industry News : http://www.industrynews.co.kr/news/articleView.html?idxno=46829

[Professor Song Min Kim and first author Kang Min Bae, from left to right]

 

On the 28th, School of EE professor Song Min Kim’s research team has announced that they have succeeded in creating the world’s first mmWave backscatter system for massive IoT connectivity.

 

The research, (OmniScatter: extreme sensitivity mmWave backscattering using commodity FMCW radar), led by Kang Min Bae as first author, was presented at ACM MobiSys 2022 this June, and was presented with the best paper award. This is meaningful as it marks the second consecutive year in which the best paper award was presented to a paper belonging to a research group at KAIST’s School of Electrical Engineering.

 

The backscatter technology described by this research team can greatly reduce the maintenance cost as it operates on ultra-lower power of less than 10 μW, being able to run on a single battery for more than 40 years.

 

By enabling connectivity on a scale that far exceeds the network density required by next gen communication technologies such as 5G and 6G, this system may serve as a great potential for serving as a stepping stone for the upcoming hyperconnected era.

 

“mmWave backscattering is a dreamlike technology that can run IoT devices on a large scale, which can drive massive communications at ultra-low power compared to any other technology,” said Professor Song Min Kim. “We hope that this technology will be actively used for the upcoming era of Internet of Things,” he added.

 

The research was made possible by the funding from Samsung Future Technology Development Project and the Institute for Information & Communication Technology Planning & Evalution.

 

 

[Fig 1. Tags used for massive IoT communications (as depicted by red triangles). Over 1100 tags are able to communicate simultaneously without any conflicts]