Archives: AI in Circuit division

[Professor Youngsoo Shin receives the October Scienc and Technology Award, Optimizing Semiconductor Process with AI]

 

<Professor Youngsoo Shin>

 

The Ministry of Science and ICT and the Korea Research Foundation announced on the 4th that Professor Shin Youngsoo from the School of Electrical Engineering at KAIST was selected as the winner of the ‘Science and Technology Award’ for October. 

Professor Shin was recognized for his contribution in developing a semiconductor lithography optimization technology that is 10 times faster and has a higher resolution than existing methods using machine learning.

 

Lithography is a process in which light is shone on a mask engraved with patterns to create devices on a wafer.

It is a critical process that determines the yield of semiconductors. In order to create polygons on the wafer, complex patterns must be drawn on the mask.

This process,  known as OPC (Optical Proximity Correction), involves repeatedly adjusting the mask shape and simulating the image on the wafer, taking a significant amount of time.

 

Professor Shin trained artificial intelligence (AI) on sets of mask shapes and the resulting wafer images to develop a faster and higher-resolution OPC optimization technique.

 Additionally, Professor Shin developed a method to create a layout pattern (semiconductor blueprint) similar in structure to existing patterns but not previously existing, using generative AI.

 

It was also confirmed that applying the newly created layout patterns and the existing sample patterns to the optimization improves the accuracy of the machine learning model.

The research results were published in the 2021 international academic journal IEEE TSM, and it also received the journal’s ‘Best Paper Award,’ which is selected once a year.

 

Professor Shin commented, “This study is unique in that it applies machine learning and artificial intelligence differently from existing semiconductor lithography research,” and added, “I hope it can contribute to resolving the issues of licensing costs and stagnation in technological development caused by the monopoly of a small number of companies worldwide.”

 

 

*Reference : 10월 과기인상에 신영수 교수…AI로 반도체 공정 최적화 (naver.com)

EE Professor Kim Joo-Young Developed A ChatGPT Core AI Semiconductor with A 2.4-fold Improvement in Price efficiency

 

 

 

The ChatGPT released by OpenAI has captured global attention, and everyone is closely observing the changes this technology will bring out.

This technology is based on large language models (LLM), which represent an unprecedented scale of artificial intelligence (AI) models compared to conventional AI.

However, the operation of these models requires a significant number of high-performance GPUs, leading to astronomical computing costs.  

 

KAIST (President: Lee Kwang-Hyung) announced that research team led by EE Professor Kim Joo-Young Kim has successfully developed an AI semiconductor that efficiently accelerates the inference operations of large language models, which play a crucial role in ChatGPT. 

The developed AI semiconductor, named the ‘Latency Processing Unit (LPU),’ efficiently accelerates the inference operations of large language models. It incorporates a high-speed computing engine capable of maximizing memory bandwidth utilization and performing all necessary inference computations rapidly.

Additionally, it comes equipped built-in networking capabilities, making it easily expandable with multiple accelerators. This LPU-based acceleration appliance server achieved up to a 50% higher performance and approximately 2.4 times better performance-to-price ratio compared to a supercomputer based on the industry-leading high-performance GPU, NVIDIA A100.

 

This advancement holds the potential to replace high-performance GPUs in data centers that are experiencing a rapid surge in demand for generative AI services. This research was conducted by Professor HyperExcel Co., founded by Professor Kim Joo-Young and achieved the remarkable accomplishment of receiving the “Engineering Best Presentation Award” at the International Design Automation Conference (DAC 2023) held in San Francisco on July 12th (U.S. time).

DAC is a prestigious international conference in the field of semiconductor design, particularly showcasing global semiconductor design technologies related to Electronic Design Automation (EDA) and Semiconductor Intellectual Property (IP).

DAC attracts participation from renowned semiconductor design companies such as Intel, NVIDIA, AMD, Google, Microsoft, Samsung, TSMC, as well as top universities including Harvard, MIT, and Stanford.

 

Among the world’s notable semiconductor technologies, Professor Kim’s team stands out as the sole recipient of an award for AI semiconductor technology tailored for large language models.

This award acknowledges their AI semiconductor solution as a groundbreaking means to drastically reduce the substantial costs associated with inference operations for large language models on the global stage.

 

Professor Kim stated, “With the new processor ‘LPU’ for future large AI computations, I intend to pioneer the global market and take a lead over big tech companies in terms of technological prowess.”

(Note: The provided translation is an elaboration and summary of the original text for clarity and readability.)

 

Professor Joo-Young Kim’s Research Team Published Article in CACM Magazine: “South Korea’s Nationwide Effort for AI Semiconductor Industry”

 

Recently, the research team led by Professor Joo-Young Kim published an article titled “South Korea’s Nationwide Effort for AI Semiconductor Industry” in CACM (Communications of the ACM), one of the leading monthly academic journals in the field of computer science.

 

 

 

EE Professor Minkyu Je is awarded the Haedong Semiconductor Engineering Award.

 

Professor Minkyu Je of our department won the “Haedong Semiconductor Engineering Award” at the 2022 regular meeting of the Semiconductor Engineering Society held at the Seoul aT Center on December 7th.

The Haedong Science and Culture Foundation has made a lot of effort and dedication to discovering and motivating talent for technological development and industry in Korea.

In particular, in 2021, the “Haedong Semiconductor Engineering Award” was established for the semiconductor industry and technology to provide a cornerstone for discovering semiconductor talent.

This year’s 2nd “Haedong Semiconductor Engineering Award” is more meaningful because it is given now that the importance of the semiconductor field is being highlighted more than ever in the world

 

[Prof. Minkyu Je]

 

Professor Minkyu Je has conducted leading research in the field of biomedical and sensor interface integrated circuits/microsystems since the early 2000s while working at the IME Research Center under A*STAR in Singapore. He was awarded because it played an important role in leading the level of domestic research in this field to the world level by continuing the highest level of research. 

Since he has played an important role in leading the level of domestic research in this field to the world level by continuing the highest level of research, he was awarded. He is recognized internationally as a major researcher who has greatly contributed to the pioneering and development of the field, and is playing a role as a leading researcher in this field in Korea.

 In recognition of excellent research achievements in the field of circuits and systems for next-generation medical devices and neuronal interfaces, he was selected as a Distinguished Lecturer by the IEEE Circuits and Systems Society (CASS) in 2020, and he presented a total of 110 SCI journal papers and 252 international conference papers.

 In particular, a total of 76 papers were published in various IEEE excellent journals related to the field, and a total of 25 papers were presented at the highest level international conferences, IEEE ISSCC, IEEE SOVC, and IEEE CICC. 

In addition, through research in this field, a total of 24 foreign patents and a total of 26 domestic patents were registered, and a total of 30 patents are pending domestic/overseas applications. 

Through active cooperation with start-ups and SMEs as well as large companies in related fields, he has significantly contributed to securing technological competitiveness in the bio/medical device and IoT sensor fields, creating and revitalizing the industrial ecosystem, and securing future growth engines targeting emerging markets. 

 

 

EE professor Joo-Young Kim receives the Science and ICT Minister’s Award

 

 

Title : Design of Processing-in-Memory with Triple Computational Path and Sparsity Handling for Energy-Efficient DNN Training

Authors : Wontak Han, Jaehoon Heo, Junsoo Kim, Sukbin Lim, Joo-Young Kim

Publications : IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), 2022

As machine learning (ML) and artificial intelligence (AI) have become mainstream technologies, many accelerators have been proposed to cope with their computation kernels. However, they access the external memory frequently due to the large size of deep neural network model, suffering from the von Neumann bottleneck. Moreover, as privacy issue is becoming more critical, on-device training is emerging as its solution. However, on-device training is challenging because it should perform the training under a limited power budget, which requires a lot more computations and memory accesses than the inference. In this paper, we present an energy-efficient processing-inmemory (PIM) architecture supporting end-to-end on-device training named T-PIM. Its macro design includes an 8T-SRAM cell-based PIM block to compute in-memory AND operation and three computational datapaths for end-to-end training. Each of three computational paths integrates arithmetic units for forward propagation, backward propagation, and gradient calculation and weight update, respectively, allowing the weight data stored in the memory stationary. T-PIM also supports variable bit precision to cover various ML scenarios. It can use fully variable input bit precision and 2-bit, 4-bit, 8-bit, and 16-bit weight bit precision for the forward propagation and the same input bit precision and 16-bit weight bit precision for the backward propagation. In addition, T-PIM implements sparsity handling schemes that skip the computation for input data and turn off the arithmetic units for weight data to reduce both unnecessary computations and leakage power. Finally, we fabricate the T-PIM chip on a 5.04mm2 die in a 28-nm CMOS logic process. It operates at 50–280MHz with the supply voltage of 0.75–1.05V, dissipating 5.25–51.23mW power in inference and 6.10-37.75mW in training. As a result, it achieves 17.90–161.08TOPS/W energy efficiency for the inference of 1-bit activation and 2-bit weight data, and 0.84–7.59TOPS/W for the training of 8-bit activation/error and 16-bit weight data. In conclusion, T-PIM is the first PIM chip that supports end-to-end training, demonstrating 2.02 times performance improvement over the latest PIM that partially supports training.

Title : T-PIM: A 2.21-to-161.08TOPS/W Processing-In-Memory Accelerator for End-to-End On-Device Training

Authors : Jaehoon Heo, Junsoo Kim, Wontak Han, Sukbin Lim, Joo-Young Kim

Publications : IEEE Custom Integrated Circuits Conference (CICC) 2022

As the number of edge devices grows to tens of billions, the importance of intelligent computing has been shifted from cloud datacenters to edge devices. On-device training, which enables the personalization of a machine learning (ML) model for each user, is crucial in the success of edge intelligence. However, battery-powered edge devices cannot afford huge computations and memory accesses involved in the training. Processing-in-Memory (PIM) is a promising technology to overcome the memory bandwidth and energy problem by combining processing logic into the memory. Many PIM chips [1-5] have accelerated ML inference using analog or digital-based logic with sparsity handling. Two-way transpose PIM [6] supports backpropagation, but it lacks gradient calculation and weight update, required for end-to-end ML training.

This paper presents T-PIM, the first PIM accelerator that can perform end-to-end on-device training with sparsity handling and support low-latency ML inference. T-PIM makes the four key contributions: 1) T-PIM can run the complete four computational stages of ML training on a chip (Fig. 1). 2) T-PIM allows various data mapping strategies for two major computational layers, i.e., fully-connected (FC) and convolutional (CONV), as well as two computational directions, i.e., forward and backward. 3) T-PIM supports fully variable bit-width for input data and power-of-two bit-width for weight data using serial and configurable arithmetic units. 4) T-PIM accelerates and saves energy consumption in ML training by exploiting fine-grained sparsity in all data types (act., error, and weight).