SHiDM: Skiplist-Based Efficient Disaggregated Memory with Hybrid Delegation (April 31)

Date: March 31, 11:45 – 12:45 pm

Abstract:

Disaggregated memory (DM) architectures separate compute from memory resources and enable flexible scaling and improving memory utilization. DM indexing techniques have been developed with hash-based, B+/radix-tree-based, LSM-tree- based, and learned/hybrid indexes. Skiplists offer an appealing alternative that is highly concurrent and scalable for building DM systems. This paper introduces SHiDM as the first skiplist-based index to build a semi-active DM through a hybrid cache and delegation model. It organizes skiplists into a hierarchical layout of wide internal nodes and dense leaf nodes, reducing the need for finegrained network transfers. Compute nodes carry out all common-case operations using one-sided RDMA, guided by lightweight, staleness-tolerant metadata and address caches that minimize repeated lookups. To handle the rare but disruptive structural modification operations (SMOS), SHIDM employs server-delegated split mechanism, where clients issue split requests and memory nodes perform SMOS locally with their limited computatio power. This hybrid delegation preserves the scalability of passive DM while avoiding the CPU bottlenecks of active designs. Our evaluation results demonstrate that SHiDM achieves up to 1.8× and 2.1x higher throughput and significantly lower tail latency than state-of-the-art DM indexes such as DEFT and CHIME respectively. Our work demonstrates that, with a hybrid delegation model and semi-active memory nodes, the skiplist offers an efficient and highly scalable indexing primitive for emerging disaggregated memory systems.

Biographical Sketch

My name is Joydhriti Choudhury, and I am currently a PhD student in the Department of Computer Science at Florida State University. I am pursuing my doctoral degree under the supervision of Dr. Weikuan Yu, whose guidance has been instrumental in shaping my research direction and academic growth. My primary research interests lie in high-performance distributed systems, with a particular focus on disaggregated memory architectures and RDMA-based key-value stores. I am especially interested in designing efficient indexing structures and caching mechanisms to reduce latency and improve scalability in large-scale data systems. Through my research, I aim to develop novel system designs that balance performance, resource efficiency, and scalability, contributing to next-generation data center architectures.

Location(In Person Only): LOV 353