Evolving State Machines as Robot Controllers

Abstract

Automated generation of robot controllers using an Evolutionary Algorithm(EA) has received increasing attention in the last years as it has the potentialfor a reduction in the development time of a robot. Often these EAs generateNeural Networks (NNs) as robot controllers. Using a NN for automaticallygenerating robot controllers has two important downsides: 1.) A human isnot able to fully understand the inner working of a multi-layer NN, and 2.)a NN has only limited abilities to decompose a complex task into sub tasks.Both of these downsides can be addressed by using a State Machine (SM)instead of a NN as robot controller. Therefore, this thesis introduces an EAcalled Evolving State Machines As Controllers (ESMAC). ESMAC generatesSMs instead of NNs. A SM is understandable for humans because ofits modularity and allows for task decomposition by using a state for eachsub task. Furthermore, two extensions of ESMAC are proposed: adaptiveESMAC and selector ESMAC. Adaptive ESMAC aims to automatically determinesthe number of states with which the best tness for a task canbe achieved. Selector ESMAC replaces the transitions that are used in aSM to switch between states with a NN-based switching mechanism. This switching mechanism allows mutations to make more gradual changes to aSM's behaviours, which improves the performance of the EA. The performance of ESMAC is evaluated on two robotic tasks: come-and-go and phototaxis-with-obstacles. All three variants of ESMAC showequally good performance as a NN-based EA on the evaluated tasks. Thecontrollers generated with standard ESMAC and adaptive ESMAC hardlymake any state transitions and mainly use one state. However, controllers that do use multiple states appear to be more robust to changing scenarios and in noisy environments. Selector ESMAC is able to generate SMs-based controllers that have complementing states and, therefore, shows potentialfor decomposing a task into sub tasks.