Graph signal processing (GSP) extends classical signal processing to signals on graphs, enabling the analysis of complex data structures through graph theory. A core challenge in GSP is graph topology identification, which aims to deduce the graph structure that best explains observed signal dependencies.

This project addresses graph topology identification for applications where the underlying structure of systems like brain and social networks is not directly observable. Traditional approaches based on signal matching and spectral templates have limitations, particularly in handling scale issues and sparsity assumptions. We introduce a novel covariance matching methodology that efficiently reconstructs the graph topology using observable data. For the structural equation model (SEM) using an undirected graph, we demonstrate that our method can converge to the correct result under relatively soft conditions. Furthermore, we extend our methodology to polynomial models, sparse directed graphs, and any known Gaussian distribution of latent variables, broadening its applicability and utility in diverse graph-based systems. Experimental results demonstrate that our method outperforms existing techniques, offering new directions for future research in graph topology identification.

Graph-based machine learning has seen significant growth during the past years with great advancements and applicability. These approaches mostly focus on pairwise interactions, neglecting the patterns of higher-order interactions which are common to complex systems. In real-world applications, we often encounter these types of signals that naturally associate with nodes, edges or sets of nodes (e.g. triangles). While the node signals have been well-studied by graph-based methods, the other signals have been researched in the recently emerging field of topological signal processing and machine learning. In this thesis, we are particularly interested in edge flow, which models the signals over the edges of a network by signal processing and learning tasks, centring on simplicial complexes. Examples of such networks can be traffic flows in a road network or water flows in a hydrological network. Recent literature in topological signal processing shows simplicial complex as a powerful and principled higher-order network model for edge flows.

In this thesis, we introduce PyTSPL, a Python library that provides reliable and user-friendly building blocks for interacting with simplicial complexes. The library aims to provide a unified platform to read network data in different formats, preprocess them and store them in a data structure such that their properties can be easily retrieved. Users can visualize the simplicial complex simply and effectively, enhancing the interpretability of complex structures and data flows. Additionally, the library provides functionality to analyze the simplicial complexes using various advanced signal processing techniques. The motivation behind developing this master’s thesis is to provide practical bridges to analysing and processing network data based on recent research methods with a unified Python library. While various tools exist for specific aspects of network analysis, there is a lack of unified platforms that integrate reading, processing, visualization, and advanced analysis of network data through topological frameworks, specifically for simplicial complexes. This library is a comprehensive solution encapsulating the entire workflow in a single environment.

This paper presents a comprehensive evaluation and comparison of encoding methods for categorical data in the context of machine learning. The study focuses on five popular encoding techniques: one-hot, ordinal, target, catboost, and count encoders. These methods are evaluated using linear models, decision trees, and support vector machines (SVMs).

The results demonstrate that one-hot encoding consistently achieves the highest accuracy across all evaluated machine learning algorithms. However, it also incurs a higher runtime, especially when feature cardinality is high. Catboost encoding emerges as a promising alternative, striking a balance between accuracy and runtime efficiency. The ordinal, target, and catboost encoders perform similarly, with small variations depending on the specific machine learning algorithm used.

Based on the findings, practitioners are advised to select one-hot encoding when accuracy is of utmost importance and computational resources are sufficient. For scenarios where runtime efficiency is critical, the catboost encoder offers competitive accuracy while minimizing training time. The ordinal encoder can be a suitable alternative when dealing with high feature cardinality.

The data used in machine learning algorithms strongly influences the algorithms' capabilities. Feature selection techniques can choose a set of columns that meet a certain learning goal. There is a wide variety of feature selection methods, however, the ones we cover in this comparative analysis are part of the information-theoretical-based family. We evaluate MIFS, MRMR, CIFE, and JMI using the machine learning algorithms Logistic Regression, XGBoost, and Support Vector Machines.
Multiple datasets with a variety of feature types are used during evaluation. We find that MIFS and MRMR are 2-4 times faster than CIFE and JMI. MRMR and JMI choose columns that lead to significantly higher accuracy and lower root mean squared error earlier. The results we present here can help data scientists pick the right feature selection method depending on the datasets used.

The curse of dimensionality is a common challenge in machine learning, and feature selection techniques are commonly employed to address this issue by selecting a subset of relevant features. However, there is no consistently superior approach for choosing the most significant subset of features. We conducted a comprehensive analysis comparing filter and wrapper techniques to guide future work in selecting the most appropriate method based on specific circumstances. We quantified the performance of these techniques using a diverse collection of datasets. We utilised simple decision trees, linear machine learning algorithms, and support vector machines to assess the performance with varying percentages of features selected by the filter and wrapper techniques. The findings demonstrate that filter methods (Chi-Squared and ANOVA) perform better than wrapper methods (Forward Selection and Backward Elimination) regarding the classification accuracy, regression root mean squared error, and runtime.

Thus far the democratization of machine learning, which resulted in the field of AutoML, has focused on the automation of model selection and hyperparameter optimization. Nevertheless, the need for high-quality databases to increase performance has sparked interest in correlation-based feature selection, a simple and fast, yet effective approach to removing noise and redundancy in relational data. However, little to no attention has been paid to what correlation metric to choose in order to maximize the performance of ML systems. Our research investigates the effectiveness and efficiency of four widely-known correlation measures, in particular Pearson, Spearman, Cramér's V, Symmetric Uncertainty, in a manner that simulates an AutoML-like setting. We show that the exact theoretical assumptions of the methods do not always hold in practice, as well as shed light on the main aspects that need to be considered when integrating correlation-based feature selection in ML systems. Notably, the results indicate that the performance obtained by correlation-based methods is highly tied to the types and number of features present in the underlying database rather than the choice of ML algorithm. We devise promising conclusions that can further serve the advancement of AutoML systems by making feature selection fully automatic and computationally tractable.

The Hierarchical Subspace Iteration Method is a novel method used to compute eigenpairs of the Laplace-Beltrami problem. It reduces the number of iterations required for convergence by restricting the problem to a smaller space and prolonging the solution as a starting point. This method has shown great performance improvements for Laplace-Beltrami eigenproblems.
We propose an adaptation to the Hierarchical Subspace Iteration Method that allows for computing vibration modes of elastic objects. We evaluate potential optimizations that can be made, as well as the performance characteristics of the method. Our method was shown to be faster than SIM in most cases while even beating Matlab's Lanczos solver in some cases.

A primary trait of humans is the ability to learn rich representations and relationships between entities from just a handful of examples without much guidance. Unsupervised few-shot learning is an undertaking aimed at reducing this fundamental gap between smart human adaptability and machines. We present a contrastive learning scheme for unsupervised few-shot classification, where we supplement a convolutional network’s strong inductive prior with a self-attention based message passing neural network to exploit intra-batch relations between images. We also show that an optimal-transport (OT) based task-awareness algorithm generates task-representative prototypes that lead to more accurate classification and aid in elevating the robustness of pre-trained models. We show that our approach (SAMPTransfer) offers appreciable performance improvements over its competitors in both in/cross-domain few shot classification scenarios, setting new standards in the miniImagenet, tieredImagenet and CDFSL benchmarks.

Earthquakes can have tremendous effects. They can result in casualties, massive damage, and hurt the economy. Therefore, one would like to predict earthquakes as early as possible and with the highest accuracy possible. This paper contains the proposal for the optimal prediction-time, which is the time between the execution of a prediction and the actual earthquake strike, for deep learning models. Only short-term predictions and high-magnitude earthquakes are considered. A prediction means to define whether an earthquake happens or does not happen in an upcoming amount of seconds. A short-term prediction means a forecast to the extent of seconds. A high-magnitude earthquake means an earthquake with a magnitude of 2.5 or higher. This research uses the Long Short Term Memory deep learning model to test the optimal prediction-time value for earthquake predictions.
The optimal value for the prediction-time is found by testing the model with different values for the prediction-time and concluding when the model performs best. For prediction-times from moments before the strike until 40 seconds, the model is performing worse compared to higher prediction-times. The model's performance peaks at a prediction-time of 70. When increasing further than 70, the performance decreases until a prediction-time of one hundred. When rising even further, the performance is stabilising. Thus, for predictions with the highest performance, one should use a prediction-time of 70 seconds.

Knowing the relation between cell types is crucial for translating experimental results from mice to humans. Establishing cell type matches, however, is hindered by the biological differences between the species. A substantial amount of evolutionary information between genes that could be used to align the species, is discarded by most of the current methods since they only use one-to-one orthologous genes. Some methods try to retain the information by explicitly including the relation between genes, however, not without caveats. In this work, we present a model to Transfer and Align Cell Types in Cross-Species (TACTiCS). First, TACTiCS uses an natural language processing model to match genes using their protein sequences. Next, TACTiCS employs a neural network to classify cell types within a species. Afterwards, TACTiCS uses transfer learning to propagate cell type labels between species. We applied TACTiCS on scRNA-seq data of the primary motor cortex and the ventral tegmental area. Our model can accurately match and align cell types on these datasets. Moreover, at a high resolution, our model outperforms two state-of-the-art methods, SAMap and CAME. Finally, we show that our gene matching method results in better matches than BLAST, both in our model and SAMap.

Water utilities face many challenges, including pipe bursts that cause significant non-revenue water losses. Detecting those bursts early is important for the water sector in its path to achieve sustainable water resource management. This study presents a scalable data-driven methodology for burst detection in water distribution systems that is based on Long Short-Term Memory (LSTM)-based neural networks (NNs) and includes two stages: prediction and classification. Time-series of hydraulic (flow and pressure) signals are fed to the LSTM, whereas domain (time) features of the next time step are fed independently to regular neurons. These two streams of information are then concatenated to predict the values of the hydraulic features of the next time step. The model is trained on normal conditions only, so that when fed with data corresponding to a burst, the predictions will mismatch the observations. Comparison of the predictions to the observations is quantified though an error function, which is then used for classification. Specifically, a variable error threshold that corresponds to a pre-defined extreme percentile of the error distribution is used to discern bursts from normal conditions. The methodology is corroborated on two different types of bursts: (a) real bursts in district metered areas (DMAs) in the United Kingdom and (b) simulated fire hydrant leak tests in the same DMAs. For the real bursts, sensitivity analysis of the algorithm is performed to assess how data resolution and error threshold affect the performance. The flexibility of the method is studied for the simulated fire hydrant leaks, where additional information streams from new sensors are incorporated in the model by means of applying transfer learning and fine-tuning. The results obtained demonstrate that this scalable LSTM-based methodology works reasonably well in real-life settings and can successfully identify burst events, both real and simulated, even in DMAs with a small number of installed sensors. Furthermore, it is assessed how the flexibility of the LSTM neurons is pivotal for burst detection when utilizing a varying number of sensors.

Humans make decisions when presented with choices based on influences. The Internet today presents people with abundant choices to choose from. Recommending choices with an emphasis on people's preferences has become increasingly sought. Grundy (1979), the first computer librarian Recommender System (RS), provided users with book recommendations. Growing volumes of user data in the '90s saw increased usage of commercially available RS for e-commerce, music, movies, books, and social networking services. Due to their effectiveness in providing recommendations, Collaborative Filtering (CF) algorithms are predominantly used to build these RS. However, traditional CF algorithms adopting Matrix Factorization (MF) and Nearest Neighbor (NN) methods suffer from handling sparse data or model scalability. With exponentially increasing sparse data, building scalable and accurate RS models is of focus.

This thesis uses tensors and graphs to represent available data. Emphasis is given to capturing higher-order interactions present between the data. The use of tensors is motivated as matrices cannot capture data with higher-order relations, such as variation of user ratings to items with time. The transition to using tensors has been promising with the development of efficient tensor decomposition methods and powerful machines. Graphs can capture the correlation between different entities, providing additional information intrinsic to the underlying graph structure. A Graph Regularized CANDECOMP/PARAFAC (GRCP) tensor decomposition model framework is proposed in this thesis. The thesis highlights how to graph Laplacian regularizers (GLRs) benefit CP tensor decomposition methods to build RS. The model framework is evaluated with the MovieLens data set. The model records lesser Normalized Mean Squared Error (NMSE) values than those reported in the literature. The combination of varied data sources notably aids in overcoming the drawbacks of current RS models, offering scalability with computational efficiency in linear time.

This study presents a comparison of different VariationalAutoencoder(VAE) models to see which VAE models arebetter at finding disentangled representations. Specificallytheir ability to encode biological processes into distinct la-tent dimensions. The biological processes that will be lookedat are the cell cycle and differentiation state. The cell cycleis expressed as a S- and G2M-Score and the differentiationstate is expressed as a number that quantifies the develop-ment time of the cells. First the models will be trained, afterthat the models will be evaluated. The evaluation is doneby checking the latent dimensions for a correlation with thetwo aforementioned biological processes. From this it be-came quite clear that VAE and DIP-VAE performed theworst out of the four models tested. On the other handβ-VAE andβ-TCVAE performed by far the best.

Convolutional Neural Networks (CNNs) have made significant strides in the field of image processing over the last decade. Different approaches have been taken and improvements have been suggested. This paper looks at a newer novelty to neural networks for image counting, which is based on single-pixel center localization instead of the traditional bounding boxes. This neural network’s loss function is the weighted average Hausdorff distance, which does not only take into account the number of misclassified points but also the distance between predicted points and ground truth values. The paper aims to compare the accuracy of the single-pixel center neural network on original training images of wheat heads as compared to filtered images. The filtered images have had a band pass filter applied to them, that is constructed by looking at the average frequency of wheat heads. It filters out certain lower and higher frequencies up to a threshold, and its aim is to reduce background noise and accentuate the wheat heads. Results showed that there was no significant and attributable improvement in the performance of the object counter when trained on images with filtered frequency information. A discussion of the unexpected results then carries out, with the aim of rationalizing the insignificant improvement in performance of the neural network on filtered images. As part of the discussion and conclusion, a recommendation is also made, giving insights into determining if this single-pixel center neural network is appropriate for a given dataset of images.

Wheat is among the most important grains worldwide. For the assessment of wheat fields, image detection of spikes atop the plant containing grain is used. Previous work in deep learning for precision agriculture employs the already established object detectors, Faster R-CNN and YOLO, adapted for the given context. However, these models suffer from the necessary duplicate-removal postprocessing and from the low performance on overlapping objects. On the other hand, the novel Detection Transformer (DETR) object detection approach manages to overcome such limitations, being an end-to-end anchor-free set predictor based on the transformer architecture, using the attention mechanism for modelling long-range dependencies. Consequently, the general sensitivity of this technique for small size objects in the wheat head domain is reduced. Nonetheless, previous research reflects the potential of frequency analysis techniques to increase the accuracy of a CNN. This paper aims to study the feasibility of adding frequency information as a pre-processing step to improve the performance of the DETR model for wheat head detection. Two variants of the original DETR with a mask based on the Fast Fourier Transform (FFT) of the power spectra of wheat heads and background patches are proposed and explored for improvements in prediction quality. Although promising, the best FFT-based DETR approach manages to deliver an average score of only 0.42, a slightly sub optimal performance compared to DETR’s one of 0.47. Additionally, as to grasp a sense of their capability among well-established detectors, YOLO-V3 and Faster R-CNN manage to achieve around 0.7 on the same wheat data set. Ultimately, a configurable automated overview of the development of wheat fields leads to a more efficient administration of the production process. To such end, this research explores the possible application of this new object detector in precision agriculture and provides insight into its limitations and potential ways of overcoming them.

This research paper analyses the effect that using frequency information can have on object detectors. The latter are complex networks that learn information about objects from images and are then able to predict the location of these objects in new, unseen images. There are, however, certain datasets that are hard to learn on, partly because the environment in which images are taken is diverse and complex, and also because the objects to detect can appear in fairly different shapes. The dataset considered in this paper is called the Global Wheat Head Dataset (GWHD, provided by a Kaggle competition). An object detector is run on the original GWHD images and then the performance is compared to running the detector on a frequency filtered version of the images. A mathematical transform called Fourier Transform is used to map images from their spatial (pixel) domain to a new domain called the frequency domain, where certain non-informative frequencies are filtered out and then the images are mapped back to their spatial domain. Two experiments were conducted and results show that with this specific filtering methodology, no improvement is found on the GWHD dataset using an object detector called YoloV5. A pipeline was developed which allows for custom filtering strategy implementations and customs datasets. Similar work has shown that images in their frequency domain can speed up computational time and also increase the accuracy of an object detector, so this paper also gives the opportunity for further experiments with the created pipeline.

In the oil and gas industry a crucial step for detecting and developing natural resources is to drill wells and measure miscellaneous properties along the well depth. These measurements are used to understand the rock and hydrocarbon properties and support oil/gas field development. The measurements are done at multiple times and using different tools. This introduces multiple disturbances
which are not related to physical properties of rocks or fluids themselves, and should be tackled before data is used to build subsurface models or take decisions. One important source of this disturbances is depth misalignment and in order to compare different measurements care must be taken to ensure that all measurements (log curves) are properly positioned in depth. This process is called depth matching. In spite of multiple attempts for automating this process it is still mostly done manually. This thesis addresses the automation problem and proposing a model based approach to solve it using Parametric Time Warping (PTW).

Based on the PTW, a parameterised warping function that warps one of the curves is assumed and its parameters are determined by solving an optimization problem maximizing the cross-correlation between the two curves. The warping function is assumed to have the parametric form of a piecewise linear function in order to accommodate the linear shifts that take place during the measurement process. This method, combined with preprocessing techniques such as an offset correction and low pass filtering, gives a robust solution and can correctly align the most commonly accruing examples. Furthermore, the methodology is extended to depth match logs with severe distortion by applying the technique in an iterative fashion. Several examples are given when developed algorithm is tested on real log data supplemented with the analysis of the computational complexity this method has and the scalability to larger data sets.

Visually grounded speech representation learning has shown to be useful in the field of speech representation learning. Studies of learning visually grounded speech embedding adopted speech-image cross-modal retrieval task to evaluate the models, since the cross-modal retrieval task allows to jointly learn both modalities and find their relationships. Specifically, the two modalities, i.e., audio and visual, were jointly embedded into a common space where the speech embeddings and the image embeddings were learned in this process. The obtained embeddings were evaluated by cross-modal retrieval task to see the model performance. Currently, the studies worked on visually grounded speech representation learning trained on scene-based datasets, such as Flickr8k, etc., which learn different objects to infer a new scene. The works that investigating the visually grounded speech representation model's ability to combine different attribute information to infer new objects are lacking. Therefore, this thesis presented a visually grounded speech representation model trained on the fine-grained datasets that contain high level details of objects to learn attribute information associated with objects to infer new objects. The proposed model adopted dual-encoder structure and used different DNN models to extract visual and audio features. An adapted batch loss was used to calculate the similarities between two modalities. Experiments were conducted to test the model performance: 1) The parameter adjusting to obtain a better-performed model. 2) Comparing with state-of-the-art models in speech-image cross-modal retrieval field and fine-grained text-image cross-modal retrieval field. 3) Ablation studies to evaluate components in the model. 4) Research on attention module to see its effectiveness. The results indicated that the proposed model was able to learn the relationships between attributes and objects to retrieve new visual objects and outperformed other visually grounded speech learning models.

System Dynamics (SD) is an approach to study the nonlinear behaviour of complex systems over time. SD models provide a high­level understanding of the system and aid in designing policies to achieve specific system behaviours. Conventional SD modelling requires an intensive amount of time, human resources and effort. Applying Machine Learning (ML) techniques benefits the modelling process in saving on resources. It also has the potential to provide insights into the system and prevent subjective­ ness of the modeller. This work proposes two methodologies, EvoNN and EvoESN, to learn SD models automatically for the urban system from observations under different levels of prior knowledge. EvoNN solves the automated equation formulation task for a Causal Link Diagram (CLD) and annotates it with Shallow Neural Networks (SNNs) as surrogate equations. The annotated CLD can be further used in simulating the system behaviour. We provide experimental results on a real­world urban system in Am­sterdam as well as the evaluation of the simulation results. The second methodology, EvoESN learns both the structure and the quantitative relations in the model without the prior knowledge about the structure. Trained using observation data, the EvoESN produces satisfactory results on the real­world urban system. We further incorporate the judgement from the domain expert to evaluate the learned model. Applied on a more complex system, EvoESN shows solid reliability and scalability to handle large datasets. Both EvoNN and EvoESN stand as promising supportive tools for SD modellers and remain robust even when lacking system observations.

Raccoon Serious Games develops different kinds of gaming experiences, including escape rooms. In an escape room, a group of players, usually between 2 and 20 people, are locked in a room, where they have to find clues and solve puzzles to escape. When such a room is played, there is always an operator, monitoring the progress of the players and keeping an eye on their safety. Modern escape rooms are quite technologically advanced, where all components of the room interact with each other. To run these games, Raccoon Serious Games needs a system which should manage all technical aspects within the room and provide a way for the operator to keep track of the players. Furthermore, the system must be flexible enough to handle many different escape rooms, which are hardcoded in configuration files. To meet these needs of Raccoon Serious Games, a team of three developers has developed a new system, called S.C.I.L.E.R.. In ten weeks, they have created a system from scratch. First, they researched technologies and requirements that would be necessary for the system and afterwards implemented the system based on their findings. S.C.I.L.E.R. allows an operator to monitor a complete escape room. The system connects a user interface to all the devices in the room, which are controlled by the user as well as the configuration of the escape room. This configuration is generated from a JSON configuration file, containing all information for the escape room. The system allows the operator to send hints to the players, control the time and manage their progress through the status of devices in the room and the puzzles they have to solve. It also contains the feeds of cameras in the room to actually see the players. Furthermore, it provides the user with a way to check a configuration file and put it to use. The system will be used by Raccoon Serious Games in the near future for the escape rooms that they will be hosting.