Automatic schema classification for Schema-Focused Therapy using k-Nearest Neighbour

Abstract

Personality disorders affect 1 in 7 adults reducing their quality of life. Schema-focused therapy (SFT) has become very popular in Psychotherapy in the treatment of personality disorders (PD), unfortunately there is still in increasing societal need for such mental healthcare. Automation in the assessment of SFT allows for Ecological Momentary Assessments (EMA). Resulting in a dynamic assessment of schema-modes and making the treatment more socially available. Automation is realised by Allaart in the form of a conversational agent (CA), but needs a better schema classification algorithm to improve its efficacy. The goal of this study is to evaluate the k-Nearest Neighbour (kNN) algorithm along with Allaart’s dataset. The main question of the study is as follows: How well can a schema be automatically classified from a text using KNN? The method comprises of an experimentalpipeline consisting of 4 stages: Labeling of dataset; pre-processing of the data; schema classification; and evaluation. kNN performed satisfactory in multi-label binary classification with a mean accuracy of 71% and a mean weighted f1-score of 0.62. kNN did not outperform other classification algorithm and performed inadequate in ordinal classification. Results indicate a contrast between majority and minority classes and found a recall of 100% on one of the majority classes. Hence, the data set is concluded to be imbalanced. Due to limitations on the dataset and the CA no reliable conclusion can be made on the performance of kNN in automated schema classification. This study proposed future research to conduct a field experiment
such that the CA and its ability to perform EMA is evaluated and reliable data is produced.