Counterfactual explanations are a useful tool to explain trained models. They are based on counterfactual thoughts, which are a natural human thought process that helps us reason about the past. When applied to trained models they show how to make minimal changes to a data point in order to obtain a desired output.
Most methods find these counterfactuals by optimizing a set of objectives. Previously these objectives were often combined into a loss function using an aggregation operator. This operator implicitly decides the priority between the objectives, but this ordering is not always in line with the user’s preferences.
To mitigate this the Multi-Objective Counterfactuals (MOC) method was introduced. MOC turns counterfactual generation into a multi-objective optimization problem and presents the user with a diverse set of counterfactuals that have different trade-offs for the objectives. It optimizes the set of objectives with an evolutionary algorithm called Nondominated Sorting Genetic Algorithm II.
In this thesis we optimize this problem using Multi-Objective Real-Valued Gene-Pool Optimal Mixing Evolutionary Algorithm, which is a different evolutionary algorithm. We present a single-modal method and two multi-modal methods. We compare the performance of our methods to a counterfactual generation method named Diverse Counterfactual Explanations (DiCE), which focusses on feasibility and diversity within a set of generated counterfactuals. Additionally, we also present a visualization tool for sets of counterfactuals.
The single-modal method generates counterfactuals that are realistic, but do not consistently perform well in other areas. The first multi-modal method generates diverse sets of counterfactuals, but overall performs worse. The second multi-modal method generates counterfactuals that perform similarly to the single-modal method, but are more diverse.

Object detectors have come a long way and are used for various applications. In pictures and videos, an object detector must deal with the background. In some settings, this background is indicative of the object; in others, it’s not and can even be disruptive. For models trained on data containing correlations between objects and backgrounds (background bias), it makes sense that changing the background can disrupt learned correlations. This paper is interested in how sensitive object detectors are to background changes, specifically when the training data does not contain correlations between objects and backgrounds. Models were trained on carefully controlled synthetic data, so only the backgrounds differed and correlations could be controlled. The results show that models perform better when tested with seen backgrounds than unseen backgrounds. This performance difference diminishes when the model is trained on more unique backgrounds.

In 2022, the Dutch Open Government Act (Wet open overheid, Woo) has required that government institutions share requested documents with citizens, thereby enhancing government transparency and public access to information.
However, current document retrieval processes often struggle to meet the legal requirements of the Woo, as they frequently fail to respond to requests within the legally mandated time frame due to the lengthy retrieval process.

This study addresses the technical challenges of optimizing information retrieval systems in the context of the Woo, by focusing primarily on document precision and recall.
By critically analyzing existing workflows, we identify key inefficiencies and propose enhancements.
Our research includes a comparative evaluation of dense and sparse retrieval methods to assess their effectiveness in this domain.
Additionally, we explore different preprocessing techniques, investigating their impact on retrieval performance on both sparse and dense retrieval systems, to determine the optimal approach for handling noisy, unstructured government data.

Our results show that these changes in retrieval methods can significantly improve retrieval accuracy and reduce response times.
BM25 in particular, shows strong performance, effectively handling the noisy data often present in government documents, highlighting its suitability for this context.
These insights provide insights for government institutions to improve and streamline their information retrieval workflows, and reduce delays of the Woo requests.

The tumor composition of breast cancer determines how tumors behave. Yet, there is a limited understanding of the arrangement of tumor cells in relation to cells in the tumor microenvironment (TME). In this research, we have characterized distance relationships between 324 cell-type pairs in 749 tissue samples of the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) study using Weibull distribution estimations, summarizing comprehensive spatial relationships with two parameters. The research showcased the first application of the method to a dataset of this substantial size and a dataset acquired with imaging mass cytometry. We identified distinct spatial relationships among breast cancer subtypes, particularly for basal, HER2-enriched, and luminal A tumors. The spatial relationships indicate attractive and repulsive interactions between different cell types and define cellular arrangements regardless of cellular abundance.
Moreover, several spatial relationships had significant associations with survival outcomes. Both findings could improve patient stratification and prognosis and emphasize the wealth of information that spatial analyses can retrieve. The results also confirm that Weibull distribution estimations are a suitable and effective method to summarize distance distributions. The application to other cohorts could lead to new insights into the tumor composition of different cancer types. Finally, the spatial profiling method was used to characterize neighborhoods and revealed distinct spatial relationships consistent with neighborhood characteristics but also provided new hallmarks.

Blockchain technology has allowed for the emergence of a new type of organization, the Decentralized Autonomous Organization (DAO). They have gained significant traction in recent years, reaching market capitalizations of up to 60 billion USD in 2021. These organizations coordinate economic activity by an unbounded group of people within an adversarial environment. However, despite their potential, currently deployed DAOs face notable challenges related to centralization in governance and infrastructure. This work addresses these limitations by proposing a novel architecture for a fully decentralized DAO with no compromises. We introduce a scalable governance protocol utilizing multi-signature schemes to manage shared assets effectively. To demonstrate the feasibility of our approach, we implement, deploy, and evaluate a real-world DAO called Music DAO. Music DAO serves as a compelling use case, enabling listeners to collectively invest in and listen to their favorite artists. This research represents a significant advancement in the field of decentralized organizations, with the potential to revolutionize the way people collaborate and organize themselves.

Searching the web on debated topics, which are issues under active discussion and where individual opinions diverge, can be highly challenging. It requires users to approach their queries objectively, browse many resources, and accept a certain level of uncertainty, even if it conflicts with personal values. In particular, confirmation bias, the tendency to favor information that reinforces beliefs or attitudes, impacts how people gather and interpret information retrieved from search engines when searching on debated topics. It can have far-reaching effects, potentially leading to conflicts, extremism, and polarization.

In an innovative approach towards mitigating the negative effects of confirmation bias on web search on debated topics, we propose implementing a boosting intervention aimed at enhancing Intellectual Humility (IH) – an individual’s ability to acknowledge the fallibility of one’s own beliefs and the limits of one’s knowledge while remaining open to learning from others’ perspectives even if they differ from their own viewpoint. While previous research has highlighted the potential benefits of boosting IH as a means to mitigate confirmation bias, its impact on users’ search behavior has yet to be explored.

Our work bridges this gap through two randomized preregistered user studies, gaining valuable insights into the effectiveness of IH-boosting interventions in mitigating confirmation bias. In the first study, we assessed the effect of three boosting interventions with different levels of complexity on users’ context dependent IH. In the second study, we examined the effects of these interventions on web search behavior.

The first experiment successfully demonstrated the effectiveness of the interventions in boosting participants’ IH across all three treatment groups. However, applying these interventions to web search, no significant differences in search behavior were observed. Our exploratory findings reveal that both individual and environmental factors, including occupation, personal viewpoints, and search results order, shape the impact of IH-boosting interventions on online search behavior, with varying effects observed across different debated topics. We hope this study inspires, and is an initial basis for continued efforts to explore the multifaceted relationship between IH, information-seeking behavior, and responsible opinion formation, ultimately promoting a more informed and unbiased online discourse.

There is a correlation between music and affect which researchers try to define and use in technology to improve healthcare and users' experience in music-related technology. However, since affect is a complex term there is not a specified way on how to represent different affective states in systems. A systematic literature review was performed to give an overview of affect representation schemes (ARS) used in music affect prediction. All the studies included in this survey were found in Scopus, Web of Science and IEEE Explorer and focus on automatic affect prediction in music. What is more, the literature written in a different language than English and not from the Computer Science field was excluded. Considering the time constraint of 10 weeks, an additional feasibility filtering was conducted. That is only the documents which mention common benchmark datasets for this task were included. The datasets were chosen after the research of related work and different literature reviews related to this topic. We were aware that it might include a bias correlated to dataset popularity. At the final stage of filtering, 113 records were chosen for data extraction from which 51 were used in the analysis due to time constraints and a single person conducting this research. After synthesising and analysing the extracted data we observed that the schemes can be distinguished between dimensional and categorical approaches and both are similarly popular. A rare but existing ARS are developed by combining both scheme types in a unique representation. What is more, there is no easily visible trend over time in the usage of certain schemes and only 66% of the reviewed studies support their choices with psychological theories. While conducting this research, we encountered limitations in terms of time and the research group, so we leave a gap for future improvements in this project.

Affective Video Content Analysis aims to automatically analyze the intensity and type of affect (emotion or feeling) that are contained in a video and are expected to arise in users while watching that video. This study aims to provide a systematic overview of various affect representation schemes utilised by researchers in the field of Affective Video Content Analysis and the reasons behind their choice. The main objectives of the study were to investigate the diversity of affect representation scheme types, their popularity over time, the basis of their selection, and the relationship between input data sources, in terms of direct and implicit analysis, and scheme types. Following the PRISMA guidelines to conduct a systematic literature review, a total of 45 papers were included in the study which were original journals and conference proceedings in the field of Affective Video Content Analysis and that were related only to Affective Movie Content Analysis published in English after 2008. Papers concerning Video Emotion Recognition were excluded from the review. The findings reveal that dimensional, categorical, and combined approaches are commonly used in this field, with the dimensional approach based on valence and arousal being the most prevalent. However, there is no significant trend in the popularity of affect representation scheme types over time. The study highlights the lack of clear motivation and explicit justifications for scheme selection, emphasizing the need for transparency and the inclusion of psychological theories as a basis for scheme choices. Additionally, the study found that audiovisual data was the most commonly used input compared to physiological signals and visible behavioural data. The study acknowledges its limitations, including time constraints and single researcher involvement, and suggests allocating more time and involving multidisciplinary teams for comprehensive insights.

Automatic affect prediction systems usually assume its underlying affect representation scheme (ARS). This systematic review aims to explore how different ARS are used for in affect prediction systems based on spoken input. The focus is only on the audio input from speakers. Various datasets for speech emotion recognition were also involved in the study to understand the motivation for certain (categorical or dimensional) schemes used for emotions. The basis, popularity, advantages and target affective states were investigated. We used Scopus and Web of Science to extract the papers, focusing on the systems in the field of Computer Science in English language. In summary, our exploration of affect representation schemes in Speech Emotion Recognition (SER) reveals a predominant focus on categorical representations of affect, particularly variations of Ekman's six basic emotions. Behavior and attitude, although rare, are also represented sometimes. Emotions like anger, happiness, and sadness receive the most attention, while the recognition of the neutral state as an emotional state remains controversial. Dimensional affect representation schemes are less common, possibly due to the difficulty in estimating valence solely from audio input. Researchers often combine multiple categorical schemes to accommodate different datasets used in SER systems, aligning the popularity of the schemes with the corresponding datasets. However, issues such as a lack of explanation for chosen categories, interchangeable use of terminology, and a weak psychological foundation for category selection pose challenges in achieving a comprehensive understanding of affect representation in SER research.

Physiological signals, such as Electroencephalogram (EEG), Glavic Skin Response (GSR), or Body Temperature, are common inputs for Automatic Affect Recognition (AAR) systems. One of the crucial elements of AAR is the Affect Representation Scheme (ARS) used to define the affective states recognized by the system (e.g., happiness, sadness, fear, anger). Throughout the years many AAR reviews have been published. However, most of them do not include a detailed analysis of ARSs and the motivation behind them. This paper aims to fill this knowledge gap by performing a Systematic Literature Review (SLR) of Computer Science papers that propose a Physiological-signal-based AAR (PAAR) system. We explore how researchers discuss and choose an ARS and whether they base it on actual psychological theories. Eligible papers are retrieved from 4 databases: Web Of Science, IEEExplore, Scopus, and ACM Digital Library. Due to time limitations, the review is done rapidly and some additional search constraints are applied for feasibility. The most significant restrictions are: considering papers published since 2020 and performing experiments on specific benchmarking datasets. We take these constraints and their possible impact into consideration when interpreting the results. The presented review procedure can be stripped from the additional filters and reused for a full review. In total 115 papers are processed. The majority of papers introduce an EEG-based emotion recognition system. The analysis of the extracted information reveals that dimensional ARSs, in particular, Valence/Arousal model is the most popular. Moreover, authors often choose to reduce the dimensions to high/low values. Categorical ARSs are less frequent and usually are adopted from the dataset. Lastly, the authors do not provide extensive motivation for the choice of ARS and rarely refer to psychological theories.

In human-human interactions, the majority of information is conveyed through body language, specifically facial expressions. Consequently, researchers have been interested in improving human-computer interactions through developing systems with automatic understanding of body language and facial expressions.  This technology is especially useful due to its broad range of applications in fields such as healthcare, education, and safety & security. Vision-based automatic affect recognition (AAR) systems aim to predict a subject’s affective state based on visual input such as image or video. These systems analyze and classify subjects’ facial expressions and body language using affect representation schemes (ARS), most often classified as either categorical or dimensional. This paper explores the current state of ARS used in vision-based AAR through a systematic literature review following PRISMA guidelines. We selected 53 papers from WebOfScience according to our eligibility criteria which included computer science papers written in English proposing a vision-based AAR system targeting single subjects, and excluded studies dealing exclusively with micro-expressions or group affect recognition. Additionally, given the time limitation imposed on this research we excluded papers that were not readily accessible with our TU Delft license, used multimodal input, or did not use a dataset included in our  predefined list. For this exploration we specifically look at the schemes used, the popularity and trends of usage, motivations, and psychological basis. From the 53 reviewed papers, all of the papers target utilitarian emotions using at least one discrete ARS. The most commonly used schemes classify affective states into happiness, sadness, fear, anger, surprise, and disgust. While the majority of papers are lacking in providing explicit reasoning for their choices, most ARS are based grounded in psychological theories. Our results show an established norm within this area of research. However, they also evidence a lack of displayed critical thought in the selection of schemes. This oversight limits potential for future AAR research.

Cyber-physical systems are complex systems constructed from different independent parts. A self-driving car is an example of a cyber-physical system where independent parts have to come together in order to result in a car that is able to drive by itself. The main challenge is finding failures within the interactions between the independent parts of the self-driving system. In this paper, we present a novel algorithm REWOSA, in order to detect these faults within the self-driving software. With the use of real-world roads extracted from Google Maps combined with a multi-objective genetic algorithm, we develop a new way to generate roads for testing self-driving cars. We evaluate this algorithm against a state-of-the-art multi-objective genetic algorithm using randomly generated roads using two different setups. Our results show that REWOSA is able to generate more failures than the baseline on both the setups, as well as create more complex roads. In return, REWOSA does create a large overhead due to the complexity of the real-world roads. However, this overhead is justifiable as we can detect more faults with the more complex real-world roads.

Synthesizing audio-reactive videos to accompany music is challenging multi-domain task that requires both a visual synthesis skill-set and an understanding of musical information extraction. In recent years a new flexible class of visual synthesis methods has gained popularity: generative adversarial networks. These deep neural networks can be trained to reproduce arbitrary images based on a dataset of about 10000 examples. After training, they can be harnessed to synthesize audio-reactive videos by constructing sequences of inputs based on musical information.

Current approaches suffer from a few problems which hamper the quality and usability of GAN-based audio-reactive video synthesis. Some approaches consider only a small number of possible musical inputs and ways of mapping these the GAN's parameters. This leads to weak audio-reactivity which has a similar motion characteristic across all musical inputs. Other approaches do harness the full design space, but are difficult to configure correctly for effective results.

This thesis aims to address the tradeoff between audio-reactive flexibility and ease of attaining effective results. We introduce multiple algorithms that explore the design space by using machine learning to generate sequences of inputs for the GAN.

To develop these machine learning algorithms, we first introduce a metric, the audiovisual correlation, that measures the audio-reactivity in a video. We use this metric to train models based only on a dataset of audio examples, avoiding the need of a large dataset of example audio-reactive videos. This self-supervised approach can even be extended to optimize a single audio-reactive video directly, removing the need to even train a model beforehand.

Our evaluation of the methods shows that our algorithms out-perform prior work in terms of their audio-reactivity. Our solutions explore a wider range of the audio-reactive space and do so without the need for manual feature extraction or configuration.

The human microbiome, the ensemble of microorganisms found in and on the human body, plays a key role in human health and disease. However, the current state of microbiome analysis represents a significant challenge for machine learning algorithms. Datasets of microbiome sequences are often characterized by a regime of large dimensionality and relatively few labels, making it difficult for a model to discriminate features from random noise and avoid overfitting. It is, therefore, paramount to reduce the dimensionality of the input data while preserving their structure and information for a model to properly learn from them. K-mer frequency vectors and learnable representations through encoders are some of the embedding methods that have been proposed in literature to reduce the dimensions of the input space for machine learning algorithms operating on biological sequences. This work aims to compare how various embedding techniques influence the performance of a downstream disease detection task from microbiome sequencing data. In particular, the research shows that k-mer frequency vectors lead to better classification metrics (AUC = 0.88) compared to NeuroSEED embeddings (AUC = 0.76) on euclidean space. The work also presents how the classification problem formulation is critical to improving the overall disease detection performance.

Autoencoders seek to encode their input into a bottleneck of latent neurons, and then decode it to reconstruct the input. However, if the input data has an intrinsic dimension (ID) smaller than the number of latent neurons in the bottleneck, this encoding becomes redundant.
In this paper, we study using the Early-Bird (EB) technique, a structural pruning method, to regularise and prune the redundant latent neurons. We do this for both linear-layer and convolutional autoencoders, on 1D and 2D data. We find that increasing the strength of EB regularisation specifically on the latent layer can lead to all redundant latent neurons (and no more) being removed in one training run.
We also compare using EB in this manner to existing ID estimation methods: we find it performs comparable to older methods like local-PCA, also being relatively robust to noise, but that it does not match the best existing ID estimation methods.

Investigating changes in forest cover has been an area of intense research for decades. From manual surveys to remote sensing we have come a long way in mapping the world around us. Machine learning and its' younger sibling, deep learning, have emerged as highly useful tools on this journey. There are a wide variety of algorithms that take a stab at analysing vegetation coverage, but not all of them are easily accessible to the community at large. This was the underlying incentive for the paper you are about to read - reproducing and replicating a novel deep-learning algorithm that generates maps of forest cover in a relatively under-explored part of the world. This study lays out the methodology devised by the original authors as well as our attempts to replicate their work on a different set of data. We present the results we were able to obtain, the hurdles we encountered along the way, and a set of guidelines we feel would be helpful for future researchers.

Police officers are exposed to many potentially traumatising and stressful situations, but they do not always find the right mental health help they need. In this thesis, conversational agent Robyn is developed to help officers keep an eye on their mental well-being and find help when needed. An evaluation approach is also presented. The conversational agent applies the Stress First Aid method to determine stress indicators together with the officer.
The design of the agent followed the guidelines for Stress First Aid. The agent uses text input and output and went through an iterative design process using expert feedback. The experts provided feedback on both text and structure of the conversation. Based on their input, informal language was chosen for the presentation of the agent. The lessons for similar projects drawn from this design process are managing expectations from users, scoping broad questions, getting feedback from users on wording and not to push people when discussing suicide.
The proposed evaluation would determine if the agent is effective at improving people's awareness of their mental well-being status. This is done by comparing perceived mental well-being before and after interaction to reported mental well-being. To achieve this, a survey to measure perceived mental well-being was developed. The evaluation also provides insight in what aspects of the agent need the most improvement in future iterations.

Around the world, child helplines through their services provide a safe and confidential space for children to be heard and empowered. The Dutch Kindertelefoon is one of such helplines providing counselling services to children via call and chat all year round. In this thesis, we explore the design of a conversational agent for training counsellors of the Kindertelefoon. More specifically, we explore the design of an agent in a role-play setting where the agent acts as a child help seeker and the user, a counsellor of the helpline. We designed a conversational agent based on the Belief-Desire-Intention (BDI) model of agency that simulates a child victim of school bullying. Through interaction with the agent, a counsellor is able to learn the Five Phase Model, the conversation model that underpins the helpline's counselling methodology to ensure conversations remain child-centered. We tested a prototype based on this design with a group of counsellors at the Kindertelefoon with regards to their counselling self-efficacy and perceived usefulness of the system. Our results show that the conversational agent is able to influence the counselling self-efficacy of users, albeit a decrease in self-efficacy. The opposite would have been preferred for a learning tool to enable counsellors achieve more effective performance over time. However, feedback from participants indicate the potential of this conversational agent as an additional learning opportunity for training counsellors at the helpline.

Grid-based fluid simulations are often limited in resolution by their high memory usage and computational costs. One approach to reducing memory usage and computational costs is to vary the grid resolution over the spatial domain. We introduce DCGrid, a new data structure for fluid simulations. DCGrid is suited for instantiation in GPU memory and allows for varying resolution over the spatial domain. We developed an efficient, optimization-based method for local mesh refinement that automatically adapts the grid resolution according to user-defined parameters during simulations. Additionally, we complement our data structure with an efficient scheme for approximate handling of collisions between fluid and static solids on cells with varying resolutions. We integrate DCGrid in a cloud simulation and extend a terrain-atmosphere interaction model to work with cells of varying resolution and rapidly changing conditions. Furthermore, we demonstrate the performance of our new methods on both simple simulations of smoke flow and complex simulations of weather phenomena and compare them to similar fluid simulations on state-of-the-art adaptive grid structures.

We are living in a world full of data. Data capture the characteristics of any entity around us, like living species, properties of scientific experimentation, etc. Moreover, data provides a basis for further analysis, reasoning, or decision-making. One of the most common applications of data analysis is to group data into a set of clusters to understand the underlying structures of a given data set. The classification system can be either supervised or unsupervised, depending on whether it assigns a data object to a discrete value from a finite value list or unsupervised or unlabeled categories. In unsupervised classification, also called, clustering no labeled data is available. State-of-the-art clustering algorithms are developed in a broader sense without targeting any specific applications. Further, they are used in a variety of application domains. However, because these algorithms lack domain-specific information and user-specific input, they do not always produce relevant results. Also, categorical features in the dataset make clustering harder because they lack semantics. Moreover, unlike classification tasks that are evaluated using well-defined target labels, clustering is an intrinsically subjective task as it depends on the interpretation, need, and interest of users. The little notion of ground truth makes cluster validation harder in an unsupervised setting. Also, there is no universally adopted approach to choose features or clustering schemes. To tackle such challenges there is an increasing need for methods that engage humans in the clustering process to tailor it to specific application domains and allow it to continuously adapt to their preferences. Such an approach where we try to achieve better computation results using human knowledge lies under the hybrid intelligence domain. This thesis explores the possibility of designing an end-to-end clustering analysis workflow using hybrid intelligence. The thesis aims to answer the following research question - How can we use hybrid intelligence in cluster analysis workflow to generate user-specific clusters and evaluate them? We try to answer the research question by introducing multiple novel methods that aim to solve the following challenges: evaluating clusters, creating semantics in categorical features, and performing user-specific cluster analysis. We apply the developed methodologies on real-time financial data to cluster small and medium (SMEs) sized Dutch enterprises. By our experimentation, we can observe that we manage to cluster Dutch SMEs as per the user-specific goals based on their financial behaviors. We believe that we achieve such results by creating semantics in the categorical features. The clustering results are further analyzed from a user requirement perspective. Our proposed cluster validation game enables us to validate cluster objects using human intelligence. The associated experimentation results give us confidence in our hypothesis that hybrid intelligence is one of the solutions to solve the clustering challenges.