PRECLUDE: PRivacy-prEserving Collaborative Learning Using a Decentralised Ensemble approach

Abstract

Machine learning techniques receive significant responsibilities, despite growing privacy concerns. Early-stage autonomous vehicles are increasingly appearing on the streets, carrying the burden of transporting human-lives to their destination. Meanwhile, doctors are involving Artificial Intelligence (AI) in their medical diagnoses, basing treatment of patients on the analyses AI provides. For these services to reach their full potential, a vast amount of training data is required, often gathered from a variety of sources. In many cases, the required data is considered to be privacy-sensitive (e.g., medical data). Due to the sensitivity of the underlying information, many individuals and organizations are not willing to entrust its protection to another party.
A field that attempts to limit the need to transfer training data openly is called collaborative learning, where multiple data generators cooperate to jointly train a classifier. In the proposed techniques the participants aim to limit the privacy loss of their collected training data to other collaborators. We contribute a clear overview of the current state-of-the-art and identify its limitations. Based on these limitations, we present two innovative protocol designs that pave the way towards private collaborative learning.
The ECONoMy protocol is developed to suit the needs of a high participant use case (i.e., Internet of Things (IoT)), under an assumed semi-honest adversarial model. The experimental results show that ECONoMy offers the desired privacy properties while remaining competitive to the non-privacy preserving alternative with which it is compared. However, in certain environments the incentives can grow exceedingly large rendering the ’semi-honest’ adversary assumption impractical.
We, therefore, created the PRECLUDE protocol which uses traceable ring signatures to protect against adversaries in the covert adversarial model. The tracing capability allows to detect malpractice and leak the identity of the deviant while preserving the anonymity of honest participants. These additional privacy-preservation properties came at a high cost to the overall efficiency, which is what we aimed to reduce by designing our extended protocol called PRECLUDE+.
PRECLUDE+ manages to drastically improve efficiency by reducing the number of participants included in a single signature. Further, we created a batch-verification phase that allows us to omit several exponentiations in each execution. We provide a detailed statistical analysis showing how to balance the efficiency improvements, with the required privacy parameters. The protocols presented in this thesis significantly improve upon the privacy guarantees offered by current alternatives, and provide a clear direction in which future work can continue to build.