Deep Learning performance dependents on the application and methodology. Neural Networks with convolutional layers have been a great success in multiple tasks trained under Supervised Learning algorithms. For higher dimensional problems, the selection of a deep network architecture can significantly improve the accuracy of the network, however for low dimensional problems this might not be true. Shallow Neural Networks have successfully matched the performance of Deep Neural Networks in multiple tasks in the past and have been shown to be expressive enough to represent low dimensional continuous control problems. Through the thesis, the performance and expressiveness of Shallow and Deep networks is compared for low-dimensional continuous control tasks. The thesis begins by comparing the two network architectures in a Supervised Learning algorithm and progresses towards state-of-
the-art Reinforcement Learning algorithms. The thesis provides an empirical approach
towards comparison of neural networks and makes conclusions that can support the selection of a network architecture for continuous control applications using Deep Reinforcement Learning algorithms.