Transformer-Based Cache Replacement Policy Learning

Cache replacement policy is critical in computer system. It determines which data to be evicted from the cache when new data is coming. A good cache replacement policy increases the cache hit rate and decreases system delay significantly. There are a few heuristic methods designed for specific access patterns, but they perform poorly on diverse and complex access patterns. In order to deal with complicated access patterns, we formulate the cache replacement problem as matching question answering and design a Transformer-based cache replacement (TBCR) model. TBCR learns access patterns based on a Transformer encoder, and this architecture performs well even on complex access patterns. We evaluated on six memory-intensive Standard Performance Evaluation Corporation (SPEC) applications. TBCR increases cache hit rates by 3 $$\%$$ over the state-of-the-art.