Annotation Practices in Machine Learning Research On Depression

Abstract

Depression diagnosis and treatment remain difficult tasks that could be improved with machine learning models. But those automatic systems should be reliable to apply in clinical psychology settings. Performing predictions in this field is most commonly done using supervised learning models, which rely on well-established annotations. Therefore this paper examines annotation practices in research surrounding depression and provides a perspective on the quality of established methods. Firstly, 80 papers were surveyed in terms of reported annotation practices. Then the results were collected and analyzed. The findings suggest that papers from the Computer Science domain would benefit from the utilization of expert knowledge and better practices of human verification. While papers across all domains missed important information on the annotation process and rarely (20\% of papers) provided input data.