Stride-Aware Page Prefetching for GPU Unified Memory via Markov Pattern Prediction

Abstract

GPU unified memory simplifies programming by automatically migrating pages between CPUs and GPUs, but page faults trigger migrations with hundreds of microseconds to millisecond-scale latency, stalling thousands of threads. We target this bottleneck with a page prefetching framework that predicts future faults by modeling stride (delta) transitions in addition to addresses. Across GPU workloads, we observe that while individual fault addresses may be unique, the sequence of strides between them exhibits strong regularity, including recurring multi-step and oscillatory patterns.