Improving Existing Optimal Decision Trees Algorithmsby Redefining Their Binarisation Strategy

Abstract

Optimal decision trees are not easily improvable in terms of accuracy. However, improving the pre-processing of underlying dataset can be the answer to creating more accurate decision trees. In this paper, multiple methods of binarising datasets are considered and the resulting decision trees compared. The binarisation is divided into two stages: discretisation and encoding, with various algorithms considered for both of the stages. Additionally, processing the data during the decision tree building, referred to as online processing, instead of beforehand, was considered. It was discovered that for smaller datasets, unsupervised discretisation was preferred, and extending one-hot encoding to also consider multiple categories at once as target gave better accuracy for trees with lower depth. For bigger datasets, online processing has shown to be beneficial.