Optimizing latency for caching with delayed hits in non-stationary environment

作     者：Jiang, Bowen Yang, Yubo Jiang, Bo 

作者机构：Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai Peoples R China 

出 版 物：《PERFORMANCE EVALUATION》 (Perform Eval)

年 卷 期：2025年第169卷

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China 

主　　题：Cache algorithm Delayed hits Non-stationary Recurrent neural network Multilayer perceptron 

摘      要：Caching plays a crucial role in many latency-sensitive systems, including content delivery networks, edge computing, and microprocessors. As the ratio between system throughput and transmission latency increases, delayed hits in cache problems become more prominent. In real-world scenarios, object access patterns often exhibit a non-stationary nature. In this paper, we investigate the latency optimization problem for caching with delayed hits in a non-stationary environment, where object sizes and fetching latencies are both non-uniform. We first find that given known future arrivals, evicting the object with the larger size, a higher aggregate delay due to miss and arriving the farthest in the future brings more gains in reducing latency. Following our findings, we design an online learning framework to make cache decisions more effectively. The first component of this framework utilizes historical data within the training window to estimate the object s non-stationary arrival process, modeled as a mixture of log-gaussian distributions. Subsequently, we predict future arrivals based on this estimated distribution. According to these predicted future arrivals, we can determine the priority of eviction candidates using our defined rank function. Experimental results on four real-world traces show that our algorithm consistently reduces latency by 2%- 10% on average compared to state-of-the-art algorithms.