Before diving into the detailed architecture (network’s layers, loss function etc.) of the Neural Networks, we need to elaborate on the required manipulation of the data that will be used as input for training and validation.
Neural Network Input
As described earlier we will train and validate a Recurrent Neural Network for every selected page based on its effect on performance. We followed the
DRAM NVM
Config:
R/W Bandwidth R/W latency
DRAM capacity Config:
R/W Bandwidth R/W latency NVM capacity Page Scheduler
Page Selector
Config:
DRAM:NVM Resources for RNN Memory Trace Scheduling Period
History Page Scheduler
RNN RNN
... RNN
Config:
Migration Cost Eviction Policy
DMA
Pages to Migrate
HMS Simulator
\
HMS-Simulator Input Config
DRAM:NVM
DRAM R/W Specifications NVM R/W Specifications DMA Migration Cost DMA Eviction Policy Memory Trace Resources for RNN*
Scheduling Period*
\
HMS Simulator API (Performance Estimate)
DRAM hitrate Application Runtime
Figure 6.1: Python Profiler used to analyze the obtained Memory Access Traces
Per-Page approach and avoided making predictions across all pages for various reasons which are mentioned in section 4.
As far as the input of the RNN is concerned, we need to utilize as much information as possible in order to achieve high prediction accuracy. Unlike similar solutions that are proposed in literature [9], we cannot only use a se- quence of per-page access counts during consecutive scheduling epochs because a lot of crucial information will be thrown away. Achieving high prediction accuracy based solely on the access counts of a single page is really difficult.
To construct per-page models that have decent prediction accuracy, we need to use data that are representative of the page’s behavior in relation to the application as a whole. This was also clearly pointed out by Hashemi et al. [8], where both Program Counter and Delta information was required to obtain accurate models. On account of this, for every single Recurrent Neural Net- work we use the sequence of access counts of our page of interest combined with the sequence of access count of its 8 closest page-neighbor during consecutive scheduling epochs. This way we can utilize information that encapsulate the spatio-temporal locality of our application without inducing too much compu- tation overhead.
The input of a Recurrent Neural Network for pagex is given by the following formula
Inputi(x) =
4 X j=−4
1
|j + 1| ∗Accessesi(x+ j) (6.1) The output of the per-page Recurrent Neural Network is the predicted num- ber of accesses the page will receive during the next epoch. Our models do not need to be absolutely accurate when it comes to determining the exact amount of times a page is going to be accessed. They only need to be accurate enough so that our Page Scheduler can determine the hotness order across all pages.
Thus, there is certainly room for the prediction to slightly differ from the ac- tual number of accesses, provided that it will not influence the hotness order of the page and, by extension its placement decision on the particular scheduling interval.
We normalize the input sequence 6.1 between 0 and 1, since Recurrent Neural Network (and especially LSTMs) are performing much better in this case. Then, we denormalize the data for the final prediction. Different from [8] we do not need to make predictions over distinct integers which would increase the chance of mispredictions. This is not necessary for the purpose of our predictions, since we only need to derive information from the model about the relative hotness of a Page.
Neural Network Configuration
A simplified overview of the Neural Network’s Configuration can be seen in Figure 5.7.
Hyperparameters
The network consist of two stacked LSTM layers with 256 neurons each, fol- lowed by a Dense Layer. The history length is 20. Thus, the input data series is split in sequences of length 20, on a rolling window fashion. 34 of the dataset is used as a training dataset, while the other 14 is used for validation.
Now, a really important parameter of the Neural Network was the selection of the loss function. Using most of the popular loss function likemean squared loss resulted either in poor convergence or poor accuracy due to dataset imbalances.
Therefore, a custom weighted loss functionwas implemented that resulted in weighing each label’s contribution to the cost function inversely proportional to the frequency of the label. This is the sample code for the custom loss function
1 f r o m k e r a s i m p o r t b a c k e n d as K
2
3 def l o s s ( y_true , y_pred , w e i g h t s ) :
4 w e i g h t s = K . v a r i a b l e ( w e i g h t s )
5 y _ p r e d /= K . sum ( y_pred , a x i s = -1 , k e e p d i m s = T r u e )
6 y _ p r e d = K . c l i p ( y_pred , K . e p s i l o n () , 1 - K . e p s i l o n () )
7 l o s s = y _ t r u e * K . log ( y _ p r e d ) * w e i g h t s
8 l o s s = - K . sum ( loss , -1)
9 r e t u r n l o s s
The neural network tried to minimize this custom loss value between the predicted and actual values, using the Adam optimizer and a learning rate of 0.01. The training stops if the loss for the validation dataset is not reduced for 30 consecutive training epochs.
Implementation
As far as the implementation is concerned, our RNN models should be com- patible with our python-based Hybrid Memory System Simulation. For that reason, we use the Keras high level API [52]. We use the existing implemen- tation for the LSTM neurons, the Adam optimizer, and model training. For the additional hyperparameters that are not explicitly determined above, the default values from Keras were used. The backend execution engine used is Tensorflow.
Chapter 7
Experimental Evaluation
In this chapter, we perform an experimental assessment of the Page Scheduler in order to evaluate its performance. We previously showed (in Figure 5.1) that assuming that we had oracular knowledge of the access counts of even a small fraction of the pages, the performance improvements of an application are pretty significant. Our goal is to approach oracular knowledge of the access counts, using Machine Intelligence. Therefore, in this chapter we will not only evaluate the actual performance improvements our Page Scheduler provides, but the prediction accuracy of the per-page RNNs as well. We showcase how close to the Oracle Page Scheduler (i.e. upper limit of performance) our Page Scheduler can perform. Finally, we touch on the performance boost induced by our enhanced-LRU policy described in Section 5 and the Estimated Energy Cost of our Page Scheduling proposal.