Mastering Page Migration and Memory Management for CXL-based Tiered Memory Systems

CXL has emerged as a promising memory interface that can cost-efficiently expand the capacity and bandwidth of a memory system, complementing the traditional DDR interface. However, CXL DRAM presents 2–3× longer access latency than DDR DRAM, creating a tiered-memory system that demands an effective and efficient page-migration solution. Although many page-migration solutions have been proposed for past tiered-memory systems, they have achieved limited success. In this talk, I will demonstrate that (1) widely-used CPU-driven page-migration solutions often pick warm pages, (2) certain applications have many sparse hot pages, causing read amplification, and (3) the performance cost of identifying hot pages is sometimes high enough to degrade the performance of applications. Subsequently, I will present M5, a platform designed to facilitate the development of CXL-driven page-migration solutions, consisting of the following two key components: (1) Hot-Page Tracker (HPT) and Hot-Word Tracker (HWT) track the top-K hot pages and words, respectively, in the CXL controller, and (2) M5-manager offers software interfaces for users to synergistically use HPT and HWT with diverse policies that can identify more useful hot pages. On average, M5 can identify 31% hotter pages and offer 17% higher performance than the best CPU-driven page-migration solution, even with a simple policy.