Federated learning (FL) allows multiple clients to train a machine learning model on a server without sharing their private data. To reach a consensus, the server collects alternative information such as model updates. The sub-field of heterogeneous FL investigates scenarios where clients have varying charac-teristics. This thesis investigates the model-heterogeneity challenge, where the clients train different model architectures, leading to compatibility issues in reaching a consensus. This thesis addresses model heterogeneity in a setting where each client owns multiple devices. We also assume that devices belonging to the same client can share data among themselves. This assumption is atypical in FL and prompts us to re-assess the limitations of knowledge transfer techniques such as knowledge distillation. We propose a set of tools centered around knowledge distillation that leverage the data-sharing policy to address model heterogeneity. We also quantify the levels of data and device heterogeneity present on the devices to
develop an adaptive solution that fits the type of heterogeneity encountered. We demonstrate the effectiveness of our solution in training a machine learning model in a federated setting with model heterogeneity by comparing it with alternative solutions in terms of latency and accuracy. Finally, we provide dir-ections for further research, such as devising tests for data heterogeneity and exploring other model compression techniques in heterogeneous FL.