Federated learning for time series forecasting enables clients with privacy-sensitive time series data to collaboratively learn accurate forecasting models, e.g., in energy load prediction.
Unfortunately, privacy risks in federated learning persist, as servers can potentially reconstruct clients' training data through gradient inversion attacks.
While gradient inversion attacks are demonstrated for image, text and tabular classification tasks, little is known for time series regression tasks.
In this paper, we first conduct an extensive empirical study on inverting time series data across 4 time series forecasting models and 4 datasets, identifying the unique challenges of reconstructing both observations and targets of time series data.
We then propose TS-Inverse, a novel gradient inversion attack that improves the inversion of time series data through (i) learning a gradient inversion model that outputs quantile predictions, (ii) a unique loss function incorporating periodicity and trend regularization, and (iii) regularization according to the quantile predictions. Our evaluations demonstrate a remarkable performance of TS-Inverse, achieving at least 2x-10x improvement in terms of sMAPE metric over existing gradient inversion attacks methods on time series data.

Data in the form of tables is commonly used in the scientific and research industry, as it provides a compact, easy-to-understand and logical way of storing data. The advancement of diffusion models has significantly improved the quality of generated tabular data, but it also poses risks of misappropriation and copyright concerns. Thus, there is a need to control and monitor the data generated by diffusion models, to enable harm mitigation and protect intellectual property. This paper addresses the necessity for robust watermarking techniques specifically designed for tabular data generated by diffusion models. We propose Ellipse, a generalization of the Tree-Ring watermarking method, originally developed for square-shaped images, to handle rectangular shaped tables. We change the shape of the watermark from a circle---fit for square-shaped images---to an oval---fit for datasets of rectangle shape.
Through comprehensive experiments on four real world datasets (Abalone, Adult, Default, and Diabetes), we demonstrate that the adapted watermarking technique has a negligible drop of 3.5% in data quality, measured through correlations between real and synthetic distributions, performance of downstream machine learning tasks, and discriminability between the real and synthetic data. This is a better result than the 12.46% drop in data quality offered by having a circle mask. Ellipse introduces a non-significant average drop of 0.4% in detection efficiency compared to having a circle mask. Our implementation also offers resilience against value skewing and deletion attacks on the rows and columns of the dataset. When exposed to attacks, Ellipse has a higher Area Under the Curve (AUC) than the circular mask of Tree-Ring by an average of 7.17%. The code for Ellipse is publicly available at https://github.com/6toma/ellipse-watermark.

In the expanding field of generative artificial intelligence, the integration of robust watermarking technologies is essential to protect intellectual property and maintain content authenticity. Traditionally, watermarking techniques have been developed primarily for rich information media such as images and audio. However, these methods have not been adequately adapted for graph-based data, particularly on molecular graphs. Latent 3D graph diffusion(LDM-3DG) is an ascendant approach in the molecular graph generation field. This model effectively manages the complexities of molecular structures, preserving essential symmetries and topological features. To protect this sophisticated new technology, we adapt the Gaussian Shading, a proven performance lossless watermarking technique, to the latent graph diffusion domain. Our adaptation simplifies the watermark diffusion process through duplication and padding, making it adaptable and suitable for various message types.
We conduct several experiments using the LDM-3DG model on publicly available datasets QM9 and Drugs, to assess the robustness and effectiveness of our technique. Our results demonstrate that the watermarked molecules maintain statistical parity in 9 out of 10 performance metrics compared to the original. Moreover, they exhibit a 100\% detection rate and a 99\% extraction rate in a 2D decoded pipeline, while also showing robustness against post-editing attacks.

Tabular data is one of the most common forms of data in the industry and science. Recent research on synthetic data generation employs auto-regressive generative large language models (LLMs) to create highly realistic tabular data samples. With the increasing use of LLMs, there is a need to govern the data generated by these models, for instance, by watermarking the model output. While the state-of-the-art Soft Red List watermarking framework has shown impressive results on standard language models, it can not be seamlessly applied to models fine-tuned for generating tabular data due to i) column permutation and ii) the task’s nature of generating low entropy sequences. We propose Tabular Red GrEen LiST (T-REST), an adaptation of the Soft Red List watermarking algorithm on tabular LLMs that is agnostic to column permutation and improves detection efficiency by employing a weighted count method that favors columns with higher entropy. Our experiments on 4 real-world datasets demonstrate that T-REST introduces a nonsignificant drop of 3% in the synthetic data quality compared to the non-watermarked data, using the resemblance and downstream machine learning efficiency metrics, while achieving high detection accuracy with AUROC of over 0.98. T-REST is insusceptible to any column or row permutation and is robust against post-editing attacks on categorical columns by maintaining a True Positive Rate (TPR) of over 0.85 when 50% of categorical values are modified.

The advent of pretrained probabilistic time series foundation models has significantly advanced the field of time series forecasting. Despite these models’ growing popularity, the application of watermarking techniques to them remains underexplored. This paper addresses this research gap by benchmarking several widely used watermarking methods to time series models and by introducing a novel watermarking technique named HTW (Heads Tails Watermark). Unlike traditional probabilistic watermarking approaches, HTW uses a pseudo-random function to directly embed a signal into the numeric structure of the series, thereby greatly enhancing its robustness against potential attacks. Comprehensive experiments and evaluations reveal that on average, HTW retains 98.4% prediction accuracy, significantly outperforming other conventional LLM watermarks. Furthermore, HTW demonstrates robust performance with an average z-score of 5.28 across various datasets and attack scenarios for a series length of 48. These findings establish HTW as a superior alternative for securing pretrained probabilistic time series foundation models

Motivated by the emergence of Large Language Models (LLMs) and the importance of democratizing their training, we propose Go With The Flow, the first practical decentralized training framework for LLMs. Differently from existing distributed and federated training frameworks, Go With The Flow enables the collaborative training of an LLM on a set of heterogeneous client nodes that dedicate different resources for an undefined amount of time. Our work addresses node churn, i.e., clients joining or leaving the system, and network instabilities, i.e., network links becoming unstable or unreliable. The core of Go With The Flow is a decentralized flow algorithm that finds the most effective routing to train a maximum number of microbatches with a minimum delay. We extensively evaluate our work on LLama-like and GPT-like models, compare it against the prior art and achieve up to 45\% training time reduction in realistic and challenging scenarios of heterogeneous client nodes distributed at 10 different geographic locations with a high node churn rate. We further demonstrate resilient training in such challenging environments, without sacrificing convergence.

In many scientific fields, time series data is essen- tial, yet maintaining the integrity and legitimacy of such data is still difficult. Traditional watermarking techniques have mainly been used for multimedia. Although approaches for watermarking non-media data have been developed recently, there is still a big gap in the development of reliable and undetectable watermarking methods for time series diffusion models. We suggest a novel modification of the tree ring watermarking method for the 2D time series model LDCast, which is intended for precipitation prediction.
Through the incorporation of watermarks into the model’s process, we guarantee resilience and undetectability. Our approach preserves the LDCast model’s predicted accuracy while still being able to verifying the origins of the data. We confirm the efficacy of our method through comprehensive evaluation, underscoring its potential to improve the se- curity and integrity of time series forecasting models.

Synthetic tabular data generated by tabular generative models represent an effective means of augmenting and sharing data. It is of paramount importance to trace and audit such synthetic data, avoiding potential harms and risks associated with inappropriate usage. While watermarking techniques are increasingly used for synthetic images, little is known about how to watermark synthetic tables such that they are imperceptible for humans, detectable by algorithms, and robust against post-editing. In this paper, we present the first watermarking algorithm for tabular diffusion models, which inserts novel ripple watermarks into the latent space of tables. For every synthetic table, the watermark initiates from a central ring within
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets.

Federated learning (FL) is a privacy preserving machine learning approach which allows a machine learning model to be trained in a distributed fashion without ever sharing user data. Due to the large amount of valuable text and voice data stored on end-user devices, this approach works particularly well for natural language processing (NLP) tasks. Due to many applications making use of the algorithm and increasing interest in academics, ensuring security is essential. Current backdoor attacks in NLP tasks are still unable to evade some defence mechanisms. Therefore, we propose a novel attack, the single-character strike to address this research gap. Consequently, the following research question is posed: What are the properties of the single-character strike in a language classification task? By experimental analysis the following properties are discovered: the single-character strike is undetectable against five state-of-the-art defences, has low impact on the global model accuracy, trains slower than similar attacks, relies on characters on the edge of the distribution to function, is robust within the global model, and performs best when close to convergence and with more adversarial clients. Emphasizing its imperceptibility and persistence, the attack maintains a 70\% backdoor accuracy after a thousand iterations without training and remains undetectable against: (Multi-)Krum, RFA, Norm Clipping and Weak Differential Privacy. By providing insight into the effective single-character strike, this paper adds to the growing body of work that questions whether federated learning can be secure against backdoor attacks.

Abstract— Federated Learning (FL) makes it possible for a network of clients to jointly train a machine learning model, while also keeping the training data private. There are several approaches when designing a FL network and while most existing research is focused on a single-server design, new and promising variations are arising that make use of multiple servers, witch have the benefit of speeding up the training process. Unfortunately single-server FL networks are prone to model poisoning attacks by malicious participants, that aim to reduce the accuracy of the trained model. This work showcases the inherent resilience of the multi-server design against existing state-of-the-art attacks tailored around single-server FL, as well as propose two novel attacks that exploits multi-server topology in order to reduce the required knowledge an adversary needs to obtain to carry out the attack, while still remaining effective. Main findings are as follows: In the event that the malicious party has compromised the entire network, existing single-server attacks are sufficient to completely prevent a model from training. If they are limited to knowledge available only within the local reach of their compromised clients, the effect is minimized to where the attacks might get mitigated without any defences being necessary. However in such cases a correlation can be observed between the location of the compromised clients and the effectiveness of an attack. The novel attacks proposed in this paper exploit this relation in order to remain sufficiently effective while requiring only the same amount of data necessary for the multi-server algorithm to function.

Federated learning provides a lot of opportunities, especially with the built-in privacy considerations. There is however one attack that might compromise the utility of federated learning: backdoor attacks [14]. There are already some existing defenses, like flame [13] but they are computationally expensive [14]. This paper evaluates a version of differential privacy, where the Gaussian noise added to the aggravated model of the clipped updates is smaller than usually. This is often referred to as weak differential privacy or weakDP. This paper evaluates weakDP with different parameters to find if weakDP can be used as a defense for a language processing federated learning classifier against a backdoor attack.

Collaborative efforts in Predictive Maintenance and Control can be beneficial for manufacturers and customers in industrial environments. However, these efforts are challenged by the need for multi-dimensional sharing of information about the same type (horizontal) and piece (vertical) of equipment, privacy restrictions and the presence of heterogeneous data distributions across participants.

Existing solutions have addressed some of these challenges for forecasting or different purposes but there lacks a comprehensive approach that handles all of them for time series forecasting. To solve this problem, we introduce Time-series-based Personalized Hybrid Federated Learning (TPHFL), a hybrid federated learning (FL) strategy that combines Horizontal FL and Vertical FL to enable multi-level knowledge exchange while preserving data privacy. All participants use a personalization mechanism to make predictions that better suit their underlying data distribution.

Our approach employs a distributed model to handle vertical privacy constraints and addresses data heterogeneity across equipment through a personalisation mechanism. Through extensive experiments on four public and one industry-specific datasets, we show that TPHFL outperforms independent learning scenarios by 27.2%, providing a strong incentive for parties to collaborate.

We demonstrate the effectiveness of personalisation by showing an accuracy improvement of up to 42.7% when comparing TPHFL with personalisation to TPHFL without personalisation, and 32.7% when comparing traditional HFL methods to HFL with personalisation. Additionally, we evaluate a different configuration for personalisation and perform a detailed hyperparameter analysis to better understand the behaviour of TPHFL in different contexts.

Multi-Server Federated Learning (MSFL) is a decentralised way to train a global model, taking a significant step toward enhanced privacy preservation while minimizing communication costs through the use of edge servers with overlapping reaches. In this context, the FedMes algorithm facilitates the aggregation of gradients, contributing to the convergence of the global model. Attacks that aim to reduce the accuracy of the global model are called untargeted attacks. One such attack that is particularly difficult to detect is the Min-Max attack. This paper explores the extension of existing defenses to enhance the robustness of MSFL against the Min-Max attack.

To do this, existing state-of-the-art defenses, including Median, Krum, Multi-Krum, Trimmed-Mean, Bulyan and DnC are extended and examined for their adaptability to this context. We refer to the extended versions of these defenses as FMes-Defenses.

Our results indicate that FMes-Defenses are ineffective in preventing the Min-Max attack from diminishing the accuracy of the global model. Surprisingly, we find even FMes-DnC is inadequate despite it's Single-Server counterpart (DnC) being renowned for mitigating the Min-Max attack.

These findings emphasise the need for novel defenses specifically tailored to the nuances of MSFL. While representing a significant stride in communication efficiency, MSFL, complemented by the FedMes algorithm, may require additional measures to ensure robust security against sophisticated untargeted attacks. This research contributes valuable insights into the challenges and importance of enhancing the security of MSFL in its ongoing development.

Labels are essential for training Deep Neural Networks (DNNs), guiding learning with fundamental ground truth. Label quality directly impacts DNN performance and generalization with accurate labels fostering robust predictions. Noisy labels introduce errors and hinder learning, affecting performance adversely. High-quality labels aid convergence, optimizing DNN training towards accurate data distribution representation. Ensuring label accuracy is vital for DNNs’ effective learning, generalization, and real-world performance. Undoubtedly, ensuring the quality of labels is not only critical but also demanding, often entailing considerable resources in terms of time and cost. As the scale of datasets grows, methods such as crowdsourcing have gained traction to expedite the labeling process. However, this approach comes with its own set of challenges, most notably the inherent susceptibility to errors and inaccuracies. For example, it was observed that the accuracy of AlexNet in classifying CIFAR-10 images plummeted from 77% to a mere 10% when labels were subjected to random flips. This stark drop in accuracy exemplifies the magnitude of influence that corrupted or erroneous labels can exert on the performance of DNNs. Such instances underscore the critical relationship between accurate labels and the efficacy of DNNs in understanding and effectively leveraging data. EnsuringDNNrobustness is vital, involving strategies like noise label identification, filtering, and integrating noise patterns into training for resilientmodels. Architectural and loss function design also combats label-related challenges, enhancing DNN adaptability across applications. This thesis investigates the pivotal role of labels in DNN training and their quality impact onmodel performance. Strategies spanning noise recovery, robust learning frameworks, andmulti-label solutions contribute toDNNresilience against noisy labels, advancing both understanding and practical applications. Chapter 1 of this thesis introduces and explains the crucial elements involved in training DNNs, which include data, DNN models, and expert participation. It highlights the complexity introduced by label noise and sets the stage for the diverse methods designed in subsequent chapters to address these aspects comprehensively. @en

Effective large-scale process optimization in manufacturing industries requires close cooperation between different parties of human experts who encode their knowledge of related domains as Bayesian network models. For example, parties in the steel industry must collaboratively use their Bayesian networks on process parameters at the maker, steel properties, and application demands at the client to identify process optimizations effectively. However, business confidentiality across domains hinders collaboration, demanding alternatives to centralized inference. We propose CCBNet, the first Confidentiality-preserving Collaborative Bayesian Network inference framework. CCBNet leverages secret sharing to securely perform analysis on the combined knowledge of party models by joining two novel subprotocols: CABN, which augments probability distributions for features across modeling parties into secret shares of their normalized combination; and (ii) SAVE, which aggregate party inference result shares through distributed variable elimination. We extensively evaluate CCBNet on nine public Bayesian networks. Our results show CCBNet achieves similar predictive quality to centralized methods while preserving model confidentiality. We finally demonstrate that CCBNet scales to challenging manufacturing use cases, where involving many (16-128) parties in large networks (223-1003 features), on average, enables 45% less computation while communicating 251k values/request.

In federated learning systems, a server maintains a global model trained by a set of clients based on their local datasets. Conventional synchronous FL systems are very sensitive to system heterogeneity since the server needs to wait for the slowest clients in each round. Asynchronous fl partially addresses this bottleneck by dealing with updates once they are received. But with a single server, the system performance would be influenced if the clients are located far from the server and require very high communication costs. Another issue in single-server settings is that the client scale is limited since the server can be overloaded with heavy communication and computation workload. Moreover, a crash on the central server is fatal to the single-server system. Multi-server FLreduces the average communication cost by decreasing the distance between servers and clients. However, the bottleneck brought by the slowest clients still exists in multi-server systems that preserve synchrony, such as Hierarchical FL. The approach we follow in this paper consists in replicating the server in a way that the global training process remains asynchronous. We propose MultiAsync, a novel asynchronous multi-server FL framework that aims to address the single-server and synchronous-system bottleneck.

Vertical federated learning’s (VFL) immense potential for time series forecasting in industrial applications such as predictive maintenance and machine control remains untapped. Critical challenges to be addressed in the manufacturing industry include small and noisy datasets, model explainability, and stringent privacy requirements for training and inference of forecasting. Additionally, to increase industry adaptability, such forecasting models must scale well with the parties/clients while ensuring strong convergence and low tuning complexity. To this end, we propose and design “Secret-shared Time Series Forecasting with VFL” (STV), a novel framework with the following features: i) a privacy-preserving VFL algorithm for time series forecasters such as SARIMAX and autoregressive trees, ii) secret sharing with multi-party computation protocols for aggregating intermediate training data and for privacy-preserving serverless inference, iii) extension of secure two-party matrix operations for direct parameter optimization to multiple parties, giving strong convergence with minimal hyperparameter tuning complexity. We conduct evaluations on six diverse datasets from both public and industry-specific contexts. Our results demonstrate that STV’s forecasting accuracy is comparable to those of centralized approaches and that direct optimization can outperform centralized methods by 23.81% on forecasting accuracy, including state-of-the-art diffusion models and long-short-term memory. We also conduct scalability analysis to offer the opportunity for flexible decision-making by examining the communication costs of direct and iterative optimization, allowing navigating between these two approaches.

Face clustering is a subfield of computer vision and pattern recognition with many applications such as face recognition and surveillance. Accurate clustering of faces can also help us to create labeled datasets. However, in the domain of comics, face clustering is not well studied. Therefore, it is uncertain which methods of feature extraction and clustering perform well on faces of comic characters. In this paper, we investigate the effectiveness of comic face clustering. To conduct our investigation, we implement two pipelines: one that automatically extracts character faces from comic strips, and another that clusters the extracted faces. Using Dilbert Comics for our experiments, we examine the performance of various feature extraction and clustering methods. Additionally, we experiment with combining feature extraction methods and removing noisy samples to increase the clustering accuracy. We show that using color information is crucial for accurate clustering, and combining color with shape features further improves accuracy. However, our experiments indicate that accuracy improvement is not guaranteed for every combination of feature extraction methods. We also demonstrate that removing noisy samples using hierarchical clustering can increase clustering precision. Using our findings, we achieve an F1 score of 0.752 based on our Dilbert Comics dataset of 77,768 face images. We obtain this result by clustering 20,988 non-noisy face images into 35 clusters with a precision of 0.886.

Contrastive Language-Image Pretraining (CLIP) has gained vast interest due to its impressive performance on a variety of computer vision tasks: image classification, image retrieval, action recognition, feature extraction, and more. The model learns to associate images with their descriptions, a powerful method which allows it to perform well on unseen domains. Often, the descriptions fail to capture text which is contained within the image, a source of information which could prove useful for a handful of computer vision tasks. This limitation requires finetuning in domains where contained text is important. In fact, CLIP has mixed performance on Optical Character Recognition (OCR). This paper proposes a novel architecture: OSBC (OCR Sentence BERT CLIP), which combines CLIP and a custom text extraction pipeline, composed of an OCR model, and a Natural Language Processing (NLP) model. OSBC uses the text contained within images as an additional feature when performing image classification and retrieval. We tested the model on multiple datasets for each task, occasionally outperforming CLIP when images contained text, while maintaining finetunability, and improving the model's robustness. In addition, OSBC was designed to be generalizable, meaning it is expected to perform well on unseen domains without finetuning, though this was not achieved in practice.

With the following paper we are planning to present and explore the possibilities of the the newly introduced Poisson Flow Generative Model (PFGM). More specifically, this work aims to introduce the Conditional Poisson Flow Generative Model (CoPFGM), which by extending the existing repository of the PFGM, it will be able to be trained in a way that allows for conditional image sampling. The work aims to provide a more modular solution that can be easily adjusted for multiple data sets, including custom, as well as datasets taken directly from large Python libraries such as PyTorch and TensorFlow. Our proposed CoPFGM consists of two steps: (i) modifying the input of underlying UNet and (ii) modifying the loss function. More specifically, for (i) we have augmented the input channels of every image with one-hot-like class conditional images, and about (ii) we are introducing an updated loss function which incorporates the Cross-Entropy Loss of the generated images during training. The proposed model is tested against 2 datasets, the MNIST, and the Dilbert dataset, the latter, consists of 1100 custom images of the faces of 6 characters taken from the Dilbert Comic-Strip. The proposed model will be tested and presented in the form of an Ablation Study, with which, we show the conditional behavior of the channel augmentation, and the image improvement in terms of class representation with the Cross-Entropy loss.