Self-Adaptive Physics-Informed Neural Networks
(SA-PINNs) are a variation of traditional Physics-Informed Neural Networks (PINNs) designed to
solve the challenges of solving ”stiff” partial differential equations (PDEs). By using adaptive weighting, SA-PINNs are able to focus their attention on areas of the domain with higher errors, therefore improving accuracy. This work investigates the
roles of individual loss components, namely residuals, boundary conditions, and initial conditions, in the performance of SA-PINNs.

Physics-Informed Neural Networks (PINNs) are intended to solve complex problems that obey physical rules or laws but have noisy or little data. These problems are encountered in a wide range of fields including for instance bioengineering, fluid mechanics, meta-material design and high-dimensional partial differential equations (PDEs). Whilst PINNs show promising results, they often fail to converge in the presence of higher frequency components; a problem known as the spectral bias. Multiple studies have explored ways to overcome or minimize spectral bias specifically for PINNs. This paper builds on previous studies by investigating the impact of different gradient descent methods on the spectral bias.

Physics-Informed Neural Networks(PINNs) have emerged as a potent, versatile solution to solving both forward and inverse problems regarding partial differential equations(PDEs), accomplished through integrating laws of physics into the learning process. The applications of this new approach are endless, as these types of equations appear across numerous fields: fluids mechanics, quantum mechanics, electrochemistry and many others. Ever since their conception, researchers have continuously improved the flexibility and performance of PINNs through advancements in the architecture of neural networks, optimization algorithms, creative sampling methods and many more. As computational power and the interest of researchers grow, the revolutionary potential of PINNs is closer to fulfillment than ever. This paper aims to examine a small part of this evolutionary process, specifically the performance and flexibility of different activation functions used in the training of the PINN, as well as potential problems this approach could solve.

Today, machine learning has an accelerated impact in quantitative finance. Current models require large amounts of data, which can be expensive. A notable area of research, physics-informed neural networks (PINNs), has proven to be effective in approximating problems that are described by partial differential equations (PDEs). During training, the PDE is embedded in the loss function and evaluated at the residual points. This allows these types of neural networks to solve problems where data is scarce or noisy. Recent studies have shown that the method for sampling residual points has a great influence on training efficiency. Residual-based adaptive distribution (RAD) sampling is the adaptive sampling method used throughout this paper. This research applies PINNs with RAD sampling to solve the Black-Scholes PDE. Here, the Black-Scholes model is used to determine the price of options in a financial market. The fundamental goal of this paper is to study the difference in training performance between non-adaptive and RAD sampling. The types of options that are being considered in this study, are the European call options and the American put options. The results shown suggest that both types of options benefit from using RAD sampling compared to non-adaptive sampling. With a loss decrease of 39.33\%, American put options improve more using RAD sampling than European call options. Although European call options still show a decrease in loss of 7.57\%.

Recent studies have shown that gesture annotation schemes should account for the multidimensional nature of gestures and define their meaning in terms of referentiality and pragmatic meaning. However, accurately annotating gesture meaning in densely crowded social settings using such a coding scheme remains to be accomplished. This study uses the MultiModal MultiDimensional (M3D) labelling scheme and the EUDICO Linguistic Annotator (ELAN) tool to annotate video data from the Conference Living Lab (ConfLab) dataset. The ConfLab dataset contains 8 video recordings of standing conversations at a conference, captured from an overhead perspective, and low-frequency audio recordings of the conversations. A total of 1119 clips of individual gesture instances are generated. This data is then fed into a VideoMAE model pre-trained on the UCF101 dataset. The model achieves an overall accuracy score of 49% on the test set but shows a significant bias towards one class due to the imbalanced dataset. Due to the small size of the dataset and the similarities between gestures with different meanings, the model cannot identify different gesture types. The results demonstrate that high-frequency audio or transcripts of the conversations are vital to avoid strong and potentially incorrect assumptions when annotating gesture meaning. Further investigation is required into the annotation and classification of pragmatic meanings and Machine Learning solutions for multi-class, multi-label video classification problems.

Learning curves illustrate the relationship between the performance of learning algorithms and the increasing volume of training data [1, 2, 3]. While the concept of learning curves is well-established, clustering these curves based on fitting parameters remains an underexplored area. Our study delves into this domain and leverages the Learning Curve Database (LCDB) to discover potential patterns. We investigate whether different curve models uncover distinct patterns, examine the impact of different datasets on these learners, and explore if various learners display unique characteristics and behaviors or adhere to a common pattern. Curve model analyses conclude that most of the data points are in a single cluster (dominant cluster), indicating a potential commonality. Certain learners, such as QuadraticDiscriminantAnalysis and PassiveAggressiveClassifier, exhibit unique traits and do not conform to this common pattern, regardless of dataset attributes. Moreover, while various learners demonstrate similar characteristics within a single curve model, distinct patterns emerged when comparing across different curve models, indicating internal similarity but external divergence in behavior.

Learning curves are useful to determine the amount of data needed for a certain performance. The conventional belief is that increasing the amount of data improves performance. However, recent work challenges this assumption, and shows nonmonotonic behaviors of certain learners on certain problems. This paper presents a new approach for detecting non-monotonicity in empirical learning curves. This method monitors the degree of monotonicity violation on non-monotonic intervals, using the performance difference. In addition, the accuracy of the algorithm is being assessed through a series of diverse experiments. The proposed algorithm is applied to a subset of the extensive Learning Curve Database (LCDB). The results indicate an experimental accuracy of 95.5% in identifying non-monotonicity within real learning curves. Importantly, the metric demonstrated its ability to distinguish genuine non-monotonic trends from minor fluctuations attributed to measurement errors.

This study explores the extrapolation of learning curves, a crucial aspect in evaluating learner performance with varying dataset sample sizes. We use the Learning Curve Prior Fitted Network (LC-PFN), a transformer pre-trained on synthetic data with proficiency in approximate Bayesian inference, to investigate its predictive accuracy using the Learning Curve Database (LCDB). The assessment involves MSE as an error metric, with 2 baselines from previous studies where we see it outperform the baseline in some cases and keep on par in others. Additionally, we scrutinize instances where the LC-PFN model may exhibit shortcomings to identify trends in curve extrapolation failures, offering insights for potential modifications to the training dataset. We see a pattern in learners where LC-PFN performs consistently poorly on, whereas no significant pattern can be seen for datasets.

Analyzing food consumption patterns can provide valuable insights into the development of obesity and eating disorders. The detection and quantification of chewing strokes are essential to facilitate this analysis. One approach to food intake analysis involves evaluating chewing sounds generated during the eating process. These sounds were recorded by microphones placed to the user’s outer
ear canal. Aside from discovering the most accurate solution, the algorithms used must demonstrate sufficient efficiency to operate on low-power embedded ear-worn hardware. Three algorithms for automated chewing detection were evaluated with the help of two datasets. The first dataset consists of the food intake sounds from the consumption of three types of food. The second dataset consists of environmental noise. The data were manually labeled by recognizing mastication sounds’ visual and audio characteristics. Precision of over 80%
was achieved by all algorithms in the dataset consisting of only chewing sounds. Finally, an efficient solution has been developed to distinguish between speech and chewing sounds.

Negotiation is not a skill that comes naturally to most people. However, most people could benefit from attaining good negotiation skills. Non-verbal behaviour plays an important role in negotiations. Previous studies have shown a link between mimicry through conversational agents and self-regulation of non-verbal behaviour. The current study aims to raise a person’s self-awareness of their non-verbal behaviour through the medium of a conversational agent in negotiation training. An experiment (N = 64) is conducted where participants performed negotiations with an intermediate feedback round where the participants' non-verbal behaviour was played back on a virtual agent. The agent either mimicked the participants’ non-verbal behaviour or exhibited idle behaviour. It was found that the mimicking agent provided better feedback to the participants. Furthermore, repeated negotiations increase a person's self-awareness. The data indicate that raising a person’s self-awareness in negotiation training by utilising non-verbal mimicry through a conversational agent is feasible.

To enable communication for patients who have lost the ability to speak due to severe neuromuscular diseases, covert speech based brain-computer interfaces (BCIs) might be used. These system use neural signals arising from covert speech and translate them into text or synthesised speech. Covert speech is imagining to speak without moving any of the articulators and therefore does not rely on actual motor activity. As recognizing covert speech from neural signals is extremely challenging, machine learning algorithms are deployed. To make use of the full potential of machine learning approaches in the field of decoding covert speech and to accommodate real-world deployment of a BCI, a large number of training samples is required to train the networks.
In this study, a novel database is presented containing EEG and audio data from 20 subjects recorded during the covert and overt pronunciation of 15 Dutch prompts. To validate the recorded data, two speaker-independent classification tasks were performed using a ResNet-50 algorithm as classifier with spatial-spectral-temporal features extracted from the EEG signals. The speaker-independent three-class classification of pre-stimulus (rest) trials versus covert speech trials versus overt speech trials obtained an average accuracy of 70.6% and the speaker-independent five-class classification of five covert vowels (“aa”, “ee”, “oo”, “ie”, “oe”) obtained an average accuracy of 19.6%. Even though the five-class classification task did not reach an above chance level accuracy, the high performance reached by the three-class classification task provides support of the existence of discriminative information in the covert speech segments to decode covert speech in the future.
Future research should focus on EMG artifact detection and on determining the performance per subject to improve the dataset. Furthermore, subject normalisation strategies should be investigated to address the challenges of subject-independent covert speech decoding.

Machine learning algorithms were used in in the past decade to assist humans with recruitment and grades assessments in the academic field. For the most part, the algorithms either exacerbated existing biases or output unfair results. This could often be traced back to an ill-implementation of the systems in the social context. The academic admission process is defined as setting goals, locating candidates, ranking and accepting them. To properly integrate machine learning in such process, one may follow the Value Sensitive methodology, which suggests designing a technical system around a social value. This methodology takes into account the various stakeholders, values and technical solutions available. Later, the system should be iteratively improved and constantly evaluated and examined so that it still serves the core values as defined.

Deep learning has enabled technologies that have been perceived complex or impossible a few years ago. Deep learning models can be used to solve several complex problem statements thereby making it a prominent field of research. With the advancements of Deep learning models, their application in domains have diversified. One prominent use-case is the generation of synthetic images or videos targeting unethical media manipulation. This raises concerns on implications caused in the society where there is loss of trust in digital media. The synthetic media generated using Deep learning models are called Deepfakes (portmanteau of "Deep learning" and "fake"). Deepfake generation leverages powerful techniques using Deep learning and Artificial Intelligence (AI) to manipulate or generate visual content with a high potential prospect of deception to human perception. The application of deepfakes is primarily in face images or videos. The use-case of synthetically generated media is a powerful tool to communicate information with false motives to public. However, synthetic media generation of faces is not new with Deep learning. Prior to deepfakes, synthetic face images and videos were generated using photo-editing softwares, primarily in the computer graphics domain. The inclusion of Deep learning methods has only facilitated higher realism of synthetic media with lesser time computation required.

With synthetic media being made easier to generate using Deep learning models, appropriate detection methods should also be deployed. This thesis focuses on detecting deepfakes using Face Recognition Systems. Deepfake detection methods have been proposed and implemented since 2018. However, the fundamental challenge in deepfake detection is the capability of the model to learn discriminatory features between real and deepfake media (owing to high similarity and realism). Furthermore, the availability of real images or videos of target individuals is lesser in comparison with the number of deepfakes available thereby posing a challenge for the detection models to further learn the real vs. deepfake features.

In this thesis, a modification to traditional Siamese Network using Pairwise Learning approach is made called Anchored Pairwise Learning, to learn the similarity between a pair of input faces to address the challenges in deepfake detection with regard to availability of real faces per individuals class. Anchored Pairwise Learning combines two proposed methods called Anchor Siamese Network (ASN) and Anchor Face Detector Network (AFDN) to perform a binary classification of images belonging to real or deepfake class by anchoring one of the test input pairs. The reported best AUC performance of the method on a test set of real and deepfake images is 91.9% and 81% on an independent hold-out set. Additionally, a comparative study between the proposed method and a baseline traditional learning face recognition method is made and their results based on model performances are discussed. Furthermore, the proposed method is evaluated with other state-of-the-art deepfake detection methods from a generalization performance on the reserved independent hold-out set. The results conclude that the Pairwise learning techniques outperform traditional learning techniques in scenarios where the number of real images are lesser compared to the deepfake images.

The onset of delirium, a disturbance in the mental activities of a patient, can be potentially detected by understanding activities within an Intensive Care Unit (ICU) room. Such activities can be extracted by estimating human pose via a visual capture of the scene. This work uses a top-view depth camera in an ICU room to estimate pose of the non-patient stakeholders. The top-view leads to self-occlusions of body joints and thus poses a challenge for estimation of complete human pose. In addition, the presence of multiple persons in the room poses a secondary challenge, as detected body-joints need to be parsed into individual poses. To address these challenges, a 3D point cloud is extracted from the top-view depth image and passed through a 3D Convolutional Neural Network (CNN). This baseline method is capable of estimating both body-joints and body-parts to eventually output human pose for multiple persons. To improve the quality of output poses, the baseline method can benefit from additional spatial context since the problem of human pose estimation has a highly structured output. The proposed techniques either increase the receptive field, perform feature extraction at multiple scales or change the order of data processing. An increase in F1-score for the proposed methods highlights the importance of additional spatial context as a crucial tool to improve the performance of pose estimation models.

Recently, deep generative models have been shown to achieve state-of-the-art performance on semi-supervised learning tasks. In particular, variational autoencoders have been adopted to use labeled data, which allowed the development of SSL models with the usage of deep neural networks. However, some of these models rely on ad-hoc loss additions for training, and have constraints on the latent space, which effectively prevent the use of recent developments in improving the posterior approximations. In this paper, we analyse the limitations of semi-supervised deep generative models based on VAEs, and show that it is possible to drop the assumptions made on the latent space. We present a simplified method for semi-supervised learning which combines the discriminative and generative loss in a principled manner. Our model allows for straightforward application of normalizing flows and achieves competitive results in semi-supervised classification tasks.

Social Robotics is an emerging field in Computer Science. Most social robots currently commercially available to buy do not have fast hardware components. As a result, the built-in software has low accuracy and performance with (amongst others) speech and facial recognition and dialogs during social interaction with users. Cloud computation offers state-of-the-art techniques, performance, and accuracy with its massive available computational power, but at extra costs and increased latency.

In this work, we extend and improve a social robot's standard capabilities and performance by making use of cloud computation. This thesis covers an exploration for the trade-offs present when replacing or augmenting built-in robot software with IBM cloud services, based on the humanoid Pepper robot from Softbank.

The approach for this exploration was guided by a hospitality use case demonstrated in the offices of two companies: a Dutch Health Insurer and IBM Netherlands. Two products were developed for this single use case based on different development toolboxes. The first toolbox contains all development software from the robot's manufacturer (the NAOqi toolbox), while the second toolbox makes use of cloud services (the Watson Toolbox). Using the product built with the NAOqi toolbox, we evaluate interactions with real users and obtain baseline data and experiences. After evaluating the second product built with the Watson toolbox, we can compare differences in human-robot interaction quality, robot component quality, development methods, and software engineering complexity and Total Costs of Ownership for both products.

The main findings include an overview of relevant test metrics and test methods for a social robot's component, including acquired data for some components of the Pepper robot. We show possible architectures for a (semi) cloud-based system, and their trade-offs. Evaluations show that the cloud-based system indeed performs better and has higher human-interaction quality compared to the product built with the NAOqi toolbox, yet downsides such as latency and operating costs are present. This is also reflected in the analysis of single components, where specifically Speech-to-Text from the cloud shows a significant increase in performance and capabilities. We show that a mix of toolboxes results in the best working and cheapest social robot when considering Total Cost of Ownership. IBM Cloud pricing structures and operating costs are analyzed for this. Finally, we contribute to the currently available knowledge on this subject with a decision matrix combining all previously mentioned information in a compact form accessible to people not knowledgeable in the hospitality robot or cloud domains. With the matrix, early-development advice decisions for creating a social robot can be formulated using the data gathered in this thesis.

With a broad approach, this research focuses on finding and discussing trade-offs, rather than an in-depth analysis of all components. Providing methods to determine the data as mentioned above, findings, and trade-offs are more important than the actual numbers found in this thesis, as advances in this domain are quick and expected to change often. The end product built using the Watson toolbox is an improvement on multiple levels, yet is still not always able to autonomously and correctly finish all intended interactions. However, its capabilities, performance, and robustness are closer to the level of being used commercially. The techniques we use to extend Pepper's capabilities could be applied to any social robot.

Nowadays with the growth of social media, users upload millions of photos in different platforms online. Researchers in the field of computer vision devote their time and effort to analyze images in order to gain valuable insight. Data
analysis and classification can be impeded by different factors. One of which is the image filters that are studied in this work. People greatly change the appearance of their photos by adding filters in order to make them more appealing. Instagram is arguably one of the most popular social media platforms online. With the platform’s growth, filtering images has also become more popular. In this thesis a subset of Instagram filters has been selected in order to study their impact with a series of experiments. To our knowledge, no mention has been made of image filters’ impact in prior work, in the domain of machine classification and human perception. Image filters can create many challenges depending on the application they are used in. In this thesis, focus has been given on classification of weather conditions. Systems have been designed to receive images and accurately identify the weather conditions that exist in them solely using visual features and no prior knowledge. In weather forecasting a lot of resources are spent in order to study past and current weather conditions so as to predict the state of the weather in the future. Gathering and documenting weather related information can be aided by these aforementioned systems. However, if researchers would like to use social images to extract insight, they need to change their approach accordingly. As it is documented in the following chapters, dealing with these photos can be problematic and can cause huge decline in performance. For this reason, the algorithmic design has been changed by using different techniques inspired
from the domain of Adversarial Machine Learning to measure their effect. In addition to machine classification, filters can influence human perception as well. A study is conducted that measures the impact filters have on the ability of humans identifying the weather conditions in images. From the quantitative and qualitative analysis of the results several key findings are extracted regarding the effect of filters and the visual cues that are used by people. People have identified certain visual cues that have not been encoded in the classifier such as the type of clothing people are wearing. Instead much simpler features have been engineered and the performance of the classifier is still
quite high.

Users may show a behavioral pattern in consuming the items. For example, one might assume that a user is interested in comedy movies when this user watches comedy movies frequently. Recommender systems are designed to understand the preference of a user from his interactions with the items and suggest items that correspond to his preference. Therefore, observing users’ behavioral pattern in consuming items is essential to capture users’ interest. We define this behavioral pattern of the users in consuming the items as a user dynamic. In this thesis, we investigate user dynamic patterns/features that can be extracted from the implicit feedback and further utilize such features to further improve the recommender systems. Moreover, we aim to leverage such user dynamic features to improve the recommender systems to predict the preferred items in the far future. Three types of item features are explored to discover potential user dynamic patterns: we explore whether a user tends to consume an item that has been rated frequently by the similar users; we explore the moment when an item was rated may influence the probability that this item is rated by a user in the future; and we explore whether the popularity of an item may influence the probability that this item is rated by a user in the future. Some useful patterns are observed from the user dynamics analysis. Furthermore, we propose several
modeling approaches to integrate the observed user dynamic features into the recommender systems, and we start to integrate it into the Bayesian Personalized Ranking Matrix Factorization (BPR-MF). The results show that modeling user dynamics could improve the prediction performance of BPR-MF.

Sleep is a natural state of our mind and body during which our muscles heal and our memories are consolidated. It is such a habitual phenomenon that we have been viewing it as another ordinary task in our day-to-day life. However, owing to the current fast-paced, technology-driven generation, we are letting ourselves be sleep-deprived, giving way to serious health concerns such as depression, insomnia, restlessness, apnea and Alzheimer’s. Polysomnography (PSG) studies are used for diagnosing and treating sleep-related disorders. Although the PSG studies are considered as the gold standard, they are obtrusive and do not allow for long-term monitoring. Various wearables have been manufactured to help people monitor their sleep-health. However, these devices have been shown to be inaccurate.

The ubiquitous sensor technology employed by the wearables provides large volumes of data, recorded in the most natural setting of the user. There is an opportunity to make use of the highly available sensor data to model a sleep scoring system that could help individuals monitor their sleep-health from the comfort of their home. In this thesis, we aim to alleviate this problem by attempting to bridge the gap between the highly accurate but obtrusive medical diagnosis (PSG) and the non-intrusive yet inaccurate wearables.

In this work, we propose DeepSleep, a deep neural net-based sleep classification model using an unobtrusive BCG-based heart sensor signal. Our proposed model’s architecture uses the combination of CNN and LSTM layers to perform self-feature extraction and sequential learning respectively. We show that our model can classify sleep stages with a mean f1-score of 74% using the BCG signal. We employ a 2-phase training strategy to build a pre-trained model to tackle the limited dataset size and test the transferability of the model on other types of heart-signal. With an average classification accuracy of 82% and 63% using ECG and PPG based heart signal respectively, we show that our pre-trained model can be used in the transfer learning setting as well. Lastly, with the help of a user study of 16 subjects, we show that the objective sleep quality metrics correlate with the perceived sleep quality reported by the subjects with a correlation score of 푟 = 0.43.

Although our proposed model’s performance is not yet comparable to the medical standards, we show that it is possible to monitor our sleep-health using the wearable signals with the least domain knowledge and preprocessing techniques. The prediction and performance of our DeepSleep model show that it is able to learn the biological rules of sleep wherein it always follows a Deep or REM stage with a transitional Light stage. Our model treats the classification problem sequentially, thus, identifying important sleep parameters like the onset of sleep cycles and time spent in different sleep stages which are time-dependent factors. Furthermore, our user study, conducted using the SATED questionnaire, provides an insight into the difference in the user’s perceived sleep quality and model’s estimation. It shows that an automated classification system needs to incorporate various external factors such as environmental and ambient conditions to be able to strongly correlate with the perceived or subjective quality. We further discuss the future research gaps and opportunities that could improve the model’s performance and also extend it to other domains like irregular heart-beat and apnea detection. We consider this work to be a starting point for research into sleep and heart health using non-intrusive wearable sensors and deep neural network-based architectures.