In this thesis, we study a particle method for Bayesian deep learning. In particular, we look at the estimation of the parameters of an ensemble of Bayesian neural networks by means of this particle method, called Stein variational gradient descent (SVGD). This method iteratively updates a collection of parameters and it has the property that its update directions are chosen such that they optimally decrease the Kullback-Leibler divergence. We also study gradient flows of probability measures and show how gradient flows corresponding to functionals on the space of probability measures can induce particle flows. We formulate SVGD as a method in this space. In the regime of infinite particles we show results about convergence of SVGD. An existing convergence result for SVGD can be extended by showing that the probability measures, governing the collection of SVGD particles, are uniformly tight. We give conditions under which this holds.