Estimating model uncertainty for conditional prepayment rate predictions using artificial neural networks with dropout

Abstract

Clients with a mortgage loan may prepay a part of their loan before the contractual date. This is called prepayment. In the case of a prepayment, the bank who issued the loan earns less interest than ini- tially agreed. It is therefore essential to build accurate models for predicting prepayment behavior. In this thesis, machine learning models called artificial neural networks are used to predict conditional prepayment rates. In particular, the possibility to quantify the model uncertainty is studied. It is important to acknowledge that the accuracy of a model is not constant over its domain. Most ma- chine learning methods do not provide information about the uncertainty regarding a prediction and, unfortunately, methods that do so are often computationally expensive. This thesis studies uncertainty estimates generated by a neural network with dropout applied. Dropout is a method that randomly drops out neurons in each layer and has been suggested to prevent overfitting. We will see that this method can also be used to extract a measure of uncertainty regarding the prediction of a neural net- work efficiently. This method is used first to evaluate uncertainty for three simple functions under different distributions of the train data. Then the uncertainty estimates are evaluated for a neural network that predicts conditional prepayment rates. It will be illustrated that the uncertainty estimates provide an accurate indication of regions of the domain where predictions are inaccurate. Moreover, using a different model in the regions identified as uncertain can result in higher model performance.