Professor Sung-Ju Lee’s research team develops a smarthphone AI system that diagnoses mental health based on user’s voice and text input
Professor Sung-Ju Lee’s research team develops a smarthphone AI system that diagnoses mental health based on user’s voice and text input
CAMEL research team has been successively selected for 2023 Samsung Future Technology Development Program
The CAMEL research team from our department, led by Professor Myoungsoo Jung, has been chosen to participate in the Samsung Future Technology Development Program.
This recognition comes with support for their study titled, “Software-Hardware Co-Design for Dynamic Acceleration in Sparsity-aware Hyperscale AI.”
Large AI models, such as mixture of experts (MoE), autoencoders, and multimodal learning, grouped under the umbrella of Hyperscale AI, have gained traction due to the success of expansive model-driven applications, including ChatGPT.
Based on the insight that computational traits of these models often shift during training, the research team has suggested acceleration strategies.
These encompass software technologies, unique algorithms, and hardware accelerator layouts.
A key discovery by the team was the inability of existing training systems to account for variations in data sparsity and computational dynamics between model layers. This oversight obstructs adaptive acceleration.
To address this, the CAMEL team introduced a dynamic acceleration method that can detect shifts in computational traits and adapt computation techniques in real-time.
The findings from this research could benefit not only Hyperscale AI but also the larger domain of deep learning and the burgeoning services sector.
The team’s goals include producing tangible hardware models and offering open-source software platforms.
Samsung Electronics, since 2013, has initiated the ‘Future Technology Development Program’, investing KRW 1.5 trillion to stimulate technological innovation pivotal for future societal progress.
For a decade, they have backed initiatives in foundational science, innovative materials, and ICT, particularly favoring ventures that are high-risk but offer significant returns.
The CAMEL team has been collaborating with Samsung since 2021 on a project focusing on accelerating Graph Neural Networks (GNNs). We extend our hearty congratulations to them as they embark on this fresh exploration into the realm of Hyperscale AI.
Our department’s Professor Myounsoo Jung’s research team has developed the world’s first AI semiconductor for search engines based on CXL 3.0.
Approximate nearest neighbor search (ANNS) is widely used in commercial services such as image search, database, and recommendation systems.
However, in production-level ANNS, there is a challenge of requiring a large amount of memory due to the extensive dataset.
To address this memory pressure issue, modern ANNS techniques leverage lossy compression methods or employ persistent storage for their memory expansion.
However, these approaches often suffer from low accuracy and performance.
The research team proposed expanding memory capacity via compute express link (CXL), which is PCIe based open-industry interconnect technology that allows the underlying working memory to be highly scalable and composable at a low cost.
Furthermore, the use of a CXL switch enables connecting multiple memory expanders to a single port, providing greater scalability. However, memory expansion through CXL has the disadvantage of increased memory access time compared to local memory.
The research team has developed an AI semiconductor, ‘CXL-ANNS‘, which leverages CXL switch and memory expanders to accommodate high memory pressure that comes from extensive datasets without losing accuracy or performance.
Additionally, by using near data processing and data partitioning based on locality, the performance of CXL-ANNS is improved.
They also compared prototyped CXL-ANNS with the existing solutions for ANNS. Compared to previous research, CXL-ANNS shows 111 times higher performance. Particularly, 92 times higher performance can be achieved compared to Microsoft’s solution that is used in commercial service.
This research, along with the paper titled “CXL-ANNS: Software-Hardware Collaborative Memory Disaggregation and Computation for Billion-Scale Approximate Nearest Neighbor Search”, will be presented in July at ‘USENIX Annual Technical Conference, ATC, 2023’.
The research was supported by Panmnesia (http://panmnesia.com). More information on this paper can be found at CAMELab website (http://camelab.org).
[News Link]
The Korea Economic Daily: https://www.hankyung.com/it/article/202305259204i
The Herald Business: http://news.heraldcorp.com/view.php?ud=20230525000225
ChosunBiz: https://biz.chosun.com/science-chosun/technology/2023/05/25/4UW5LPX3WVARVIS3QBBICPINFM/
etnews: https://www.etnews.com/20230525000092
[Prof. David Hyunchul Shim]
KAIST PhD candidate Yuji Roh from the School of Electrical Engineering (advisor: Prof. Steven Euijong Whang) was selected as a recipient of the 2022 Microsoft Research PhD Fellowship.
[Yuji Roh]
The list of recipients: https://www.microsoft.com/en-us/research/academic-program/phd-fellowship/2022-recipients/
Interview (Asia): https://www.youtube.com/watch?v=qwq3R1XU8UE
[Prof. Minsoo Rhu]
[Award picture of MICRO Hall of Fame]
Related links:
MICRO: https://www.microarch.org/micro55
MICRO Hall of Fame: https://www.sigmicro.org/awards/microhof.php
[Prof. Myoungsoo Jung, Miryeong Kwon, Seungjun Lee, and Hyunkyu Cho from left]
Our department’s Professor Myoungsoo Jung’s research team has developed the world’s first Predictable Latency Mode (PLM) SSD based hardware and software co-designed framework for Log-Structured Merge Key-Value Stores (LSM KV store).
The research team has developed the ‘hardware and software co-designed framework for LSM KV store, Vigil-KV’ that eliminates long-tail latency by utilizing the Predictable Latency Mode (PLM) interface, which provides constant read latency, to the actual datacenter-scale SSD. Vigil-KV outpoerforms 3.19x faster tail latency and 34% faster average latency compared to the existing LSM KV store.
LSM KV store, a kind of database, is used to manage various application data, and it must process the user requests within the requirement time in order not to degrade the user experience. To this end, Vigil-KV enables a predictable latency mode (PLM) interface on an actual datacenter-scale NVMe SSD (PLM SSD), which guarantees constant read latency in deterministic mode related to read service without performing SSD’s internal tasks.
Specifically, Vigil-KV hardware makes the deterministic mode SSDs exist in the system to remove SSD’s internal tasks by configuring PLM SSD RAID. In addition, Vigil-KV software prevents the deterministic mode from being released by LSM KV store’s internal tasks, scheduling LSM KV store operations (e.x., compaction/flush operations) and client requests.
Among the proposed research results, especially noteworthy is that Vigil-KV is the first work that implements the PLM interface in a real SSD and makes the read latency of LSM KV store deterministic in a hardware-software co-design manner. They prototype Vigil-KV hardware on a 1.92TB datacenter-scale NVMe SSD while implementing Vigil-KV software using Linux 4.19.91 and RocksDB 6.23.0.
The KAIST Ph.D. Candidates (Miryeong Kwon, Seungjun Lee, and Hyunkyu Choi) participate in this research, and the paper (Vigil-KV: Hardware-Software Co-Design to Integrate Strong Latency Determinism into Log-Structured Merge Key-Value Stores) was reported in July, 11th at ‘USENIX Annual Technical Conference, ATC, 2022’. In addition, they has won the Best Paper Award from Samsung for this paper (Vigil-KV) with Professor Jae-Hyeok Choi’s research team.
The Best Paper Award from Samsung recognizes master’s and doctorate students that participated in research grant projects and published papers related to the project among papers adopted by foreign journals/conferences since September 21st. This year’s awards consisted of grand award (2 people), excellence award (1 person), and encouragement award (2 people).
The research was supported by Samsung. More information on this paper can be found at http://camelab.org.
EE Professor Rhu, Minsoo’s Research Team Build First-ever Privacy-aware Artificial Intelligence Semiconductor, Speeding Up the Differentially Private Learning Process 3.6 Times Google’s TPUv3
[Professor Rhu, Minsoo]
EE Professor Rhu and his research team have taken artificial intelligence semiconductors a big leap forward in the application of differentially private machine learning. Professor Rhu’s team analyzed the bottleneck component in the differentially private machine learning performance and devised a semiconductor chip greatly improving differentially private machine learning application performance.
Professor Rhu’s artificial intelligence chip consists of, among others, a cross-product-based arithmetic unit and an addition tree-based post-processing arithmetic unit and is capable of 3.6 times faster machine learning process compared with that of Google’s TPUv3, today’s most widely used AI processor.
The new chip also boasts comparable performance to that of NVIDIA’s A100 GPU, even with 10 times less resources.
[From left, Co-lead authors Park, Beomsik and Hwang, Ranggi; co-authors Yoon, Dongho and Choi, Yoonhyuk]
This work, with EE researchers Park, Beomsik and Hwang, Ranggi as co-first authors, will be presented as DiVa: An Accelerator for Differentially Private Machine Learning at the 55th IEEE/ACM International Symposium on Microarchitectures (MICRO 2022), the premier research venue for computer architecture research coming October 1 through 5 in Chicago, USA.
Professor Rhu’s achievements have been reported in multiple press coverage.
Links:
AI Times: http://www.aitimes.com/news/articleView.html?idxno=146435
Yonhap : https://www.yna.co.kr/view/AKR20201116072400063?input=1195m
Financial News : https://www.fnnews.com/news/202208212349474072
Donga Science : https://www.dongascience.com/news.php?idx=55893
Industry News : http://www.industrynews.co.kr/news/articleView.html?idxno=46829