In today’s AI-driven, data-intensive application, memory has become a serious bottleneck. Applications demand more bandwidth and capacity than conventional DRAM-only memory system architectures can deliver. Recently, Compute Express Link (CXL) memory have been attracting attention to address this issue. In particular, CXL memory is able to expand memory bandwidth, capacity as “plug-in” modules that live outside the traditional DIMM Slots.

We developed a CXL memory 2.0 expansion solution, which comprises two key components: 1) the CXL memory expander prototype(E3.s form factor, PCIe Gen 5 x8, 96GB per module) and software toolkit called the Heterogeneous Memory Software Kit (HMSDK). HMSDK offers two distinct programming models and supports a novel kernel memory allocation policy. 

We demonstrate the feasibility of our CXL memory solution by implementing it on CacheLib – an open-source, high –performance in-memory cache used in production services like CDN and social graph. CacheLib provides two caching modes tailored to different application requirements: RAM cache and hybrid cache. In RAM cache mode for high performance, only DRAM is used for cache storage, providing rapid response times for high-bandwidth workloads. On the other hand, hybrid cache expands the storage area beyond DRAM to include nonvolatile memory (NVM), thereby increasing the cache capacity and hit ratio at a relatively lower cost.

Memory System Architecture

 CXL memory in CacheLib supports two memory system architecture: 1) transparent tiering and 2) data tiering. 

1. Transparent Tiering – DRAM and CXL memory act as one unified pool. The OS automatically allocates memory pages by interleaving policy.
2. Data Tiering – An architecture that hierarchizes the memory system at the application level and stores data according to the unique capabilities of each memory device. This allows fine-grained optimization based workload patterns. 
3. 
   

Bandwidth Expansion 

To verify the bandwidth expansion effect of CXL memory solution, we applied the CXL memory solution to Cachelib's RAM Cache mode. This mode interleaves DRAM and CXL memory at a 4:1 ratio. As a result, bandwidth increased by 25%, processing performance improved by 15%, and latency decreased by 9% (“Fig. 1(a)”).

Capacity Expansion

To verify the capacity expansion effect of CXL memory solution, we applied the CXL memory expansion solution to Cachelib's Hybrid Cache mode. In this mode, CXL memory replaces DRAM. Replacing 40GB of DRAM with 40GB of CXL memory delivered almost identical performance. Expanding to 60GB or 80GB of CXL memory further boosted hit rates and throughput. We observed a 5% and 10% performance improvement compared to 40GB of DRAM (“Fig. 1(b)”).

Fig1. Get throughput tests with CXL memory expander: (a) Bandwidth Expansion, (b) Capacity Expansion

Our CXL 2.0 memory expansion solution provides server systems 1) DRAM expansion and 2) DRAM saving.

DRAM Expansion. Since CXL memory does not occupy DIMM slots, adding 4x 96-GB CXL memory modules to a single-socket server environment with 8-channel DDR5 32 GB expands bandwidth by 50% and capacity by150%.

DRAM Saving. Replacing all or part of the DRAM with CXL memory results in only 0~3% throughput loss when using data tiering. CXL memory is sufficiently competitive in caching systems where trade-offs between DRAM capacity and system performance are possible.

CXL memory is a cost-effective solution that overcomes DRAM limits, expanding both bandwidth and capacity in caching system of data center.

The Publication : Link