Introduction: Sleep deprivation is commonly encountered in critically ill children admitted to the pediatric intensive care unit (PICU) and is associated with poor clinical outcome. Automated electroencephalography (EEG)-based depth of sleep monitoring enables real-time continuous study of sleep in PICU patients without the need for visual assessment of the EEG signals, the gold standard. This study aims to evaluate the classification performance of various index measures and machine learning models for sleep monitoring in critically ill children.   Method: Two EEG-index-based approaches, calculated as the ratio gamma/delta and of gamma/(theta+delta) spectral powers, as well as three machine learning models - decision tree (DT), support vector machine (SVM) and extreme gradient boosting (XGBoost) - were trained and evaluated. The classification into three as well as four sleep states was evaluated. Polysomnography (PSG) recordings of 120 non-critically ill patients were used for model optimization, training and internal validation. As a proof-of-concept, the models were tested on the PSG data of 10 PICU patients.   Results: Whereas the machine learning models outperformed the index-measures in both three- as well as four-state classification in PSG recordings of non-critically ill children, the opposite was true for the PICU PSG data. Best results for PSG data of non-critically ill patients were obtained with the XGBoost model, with a 5-fold cross-validation accuracy of 0.79 (± 0.01) for three-state classification. Performances for PICU PSG data were remarkably worse for all models. The best results for PICU data were obtained with the index-based approach (accuracy = 0.60) and the gamma/delta and gamma/(theta+delta) performed equally. The individual assessment of model performances per PICU patient revealed large variation between them.   Conclusion: A simple index measure is a promising method to monitor sleep in PICU patients. Machine learning models developed in non-critically ill patients cannot easily be applied to PICU patients in whom the sleep EEG is frequently deviant. Future efforts should focus on further tuning, training and validating the classification models with more PICU data.