Because of the growing presence of artificial intelligence, developers are looking for more efficient methods to construct machine learning algorithms. Program synthesizers allow us to produce algorithms consisting of scalers, feature selection and classifiers. Each of these pipelines is a potential solution to the given machine learning task. The goal of this synthesizer was to find the best-suited pipeline for the problem, with a genetic search algorithm. The structure of the pipelines makes it easy to implement the cross-over and mutation properties of a genetic algorithm, as the pipelines and different algorithms very much resemble chromosomes and genes. Experiments were designed to measure the accuracies and runtimes of the synthesizer with the intent to compare them to the results of other synthesizers based on different search algorithms. The comparisons made could prove whether machine learning synthesizers are a viable solution to the mentioned development problem.

In AutoML, the search space of possible pipelines is often large and multidimensional. This makes it very important to use an efficient search algorithm. We measure the effectiveness of the Metropolis-Hastings algorithm (M-H) in a pipeline synthesis framework, when the search space is described by a context-free grammar. We also compare the performance of the M-H algorithm to other search algorithms. While AutoML frameworks use many different search algorithms, and comparisons between AutoML frameworks exist, this is the first paper that compares the performance of different search algorithms in the context of pipeline synthesis under equal conditions. We found that M-H is slightly outperformed by BFS2, the simplest possible search algorithm. We conclude that the datasets we use for evaluating the algorithms are too simple to meaningfully compare the performance of different search algorithms. We also conclude that for simple datasets, simple search algorithms work best.