Solving ML with ML: Evaluating the performance of the Monte Carlo Tree Search algorithm in the context of Program Synthesis

Abstract

Machine learning pipelines encompass various sequential steps involved in tasks such as data extraction, preprocessing, model training, and deployment. Manual construction of these pipelines demands expert knowledge and can be time-consuming. To address this challenge, program synthesis offers an automated approach to generate computer programs based on high-level specifications or examples. By leveraging program synthesis, the development of machine learning solutions can be expedited, leading to broader adaptability. A key element of program synthesis is the objective function, which guides the combinatorial search for a program that satisfies user-defined requirements. This study examines the performance of the Monte Carlo Tree Search (MCTS) algorithm in the realm of generating machine learning pipelines through program synthesis. The research investigates the method's efficacy, explores its findings in terms of accuracy, cost, variance, and execution time, and draws conclusions regarding the algorithm's potential and limitations. By analyzing the MCTS algorithm's performance, this research contributes to the advancement of automated machine learning pipeline generation and highlights the benefits and considerations associated with using program synthesis techniques.