Caiti: I/O transit caching for persistent memory-based block device

Byte-addressable non-volatile memory (NVM) sitting on the memory bus is employed to make persistent memory (pmem) in general-purpose computing systems and embedded systems for data storage. Researchers develop software drivers such as the block translation table (BTT) to build block devices on pmem, so programmers can keep using mature and reliable conventional storage stack while expecting high performance by exploiting fast pmem. However, our quantitative study shows that BTT underutilizes pmem and yields inferior performance, due to the absence of the imperative in-device cache. We add a conventional I/O staging cache made of DRAM space to BTT. As DRAM and pmem have comparable access latency, I/O staging cache is likely to be fully filled over time. Continual cache evictions and fsyncs thus cause on-demand flushes with severe stalls, such that the I/O staging cache is concretely unappealing for pmem-based block devices. We accordingly propose an algorithm named Caiti with novel I/O transit caching. Caiti eagerly evicts buffered data to pmem through CPU's multi-cores. It also conditionally bypasses a full cache and directly writes data into pmem to further alleviate I/O stalls. Experiments confirm that Caiti significantly boosts the performance with BTT by up to 3.6×, without loss of block-level write atomicity.