Efficient and effective information retrieval (IR) systems are needed to fetch a large number of relevant documents and present them based on their relevance to the input queries. Previous work reported the use of sparse and dense retrievers. Sparse retrievers offer low latency but suffer from term mismatch issues, while dense retrievers improve performance at the cost of higher processing times. The literature proposed Fast-Forward Indexes, an interpolation-based re-ranking framework that leverages the benefits of both sparse and dense retrievers.

Although a lot of work was done in the field, most studies evaluate the performance of the proposed models only on the MS Marco dataset, neglecting other datasets. This study extends previous work by exploring how different sparse retrievers, employing no-encoder, uni-encoder, and bi-encoder architectures, perform in an interpolation-based re-ranking setting on datasets originating from various domains. Results show that bi-encoder-based retrievers outperform the other sparse retrievers in terms of recall but with a substantial increase in latency compared to simpler retrievers, which generally showed good performance. Further, when the retrievers were used in an interpolation-based re-ranking setting, they performed similarly in terms of ranking quality.

The democratization of machine learning through public repositories, often known as model zoos, has significantly increased the availability of pre-trained models for practitioners. However, this abundance can make it difficult to choose the most suitable pre-trained model for fine-tuning on new tasks. Although various methods have been proposed in the field of transferability estimation to address this issue, these methods can take hours to execute and may still fail to find the optimal pre-trained model for fine-tuning. By exploring a new graph learning-based approach to transferability estimation, we outperform state-of-the-art methods such as LogME, improving the accuracy of the best-predicted model by up to 31.5\% in less than 5 minutes.

Synthetic tabular data generated by tabular generative models represent an effective means of augmenting and sharing data. It is of paramount importance to trace and audit such synthetic data, avoiding potential harms and risks associated with inappropriate usage. While watermarking techniques are increasingly used for synthetic images, little is known about how to watermark synthetic tables such that they are imperceptible for humans, detectable by algorithms, and robust against post-editing. In this paper, we present the first watermarking algorithm for tabular diffusion models, which inserts novel ripple watermarks into the latent space of tables. For every synthetic table, the watermark initiates from a central ring within
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets.

Most of the adversarial attacks suitable for attacking decision tree ensembles work by doing multiple local searches from randomly selected starting points, around the to be attacked victim. In this thesis we investigate the impact of these starting points on the performance of the attack, and find that the starting points significantly impact the performance: some do much better than others. However, we do find that this is not the case for all attacked points, as there are large differences between points in how difficult they are to attack and for all datasets some points are always optimally attacked.

We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.

Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.

Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly.

Bilateral teleoperation with force feedback aims to transmit human expertise over long distances by transferring the sensation of physical contact. One of the primary challenges in achieving this goal is the ultra low latency requirement. Tactile internet and model-mediated teleoperation are promising research areas addressing the latency constraint. In model-mediated teleoperation, it is crucial that the remote environment parameters are tracked with minimal latency to update the local model. This work investigates the potential of designing a cost-effective object tracking solution that performs comparably to state-of-the-art systems while maintaining low tracking latency. A method is proposed to streamline the design process by leveraging assumptions about the tracking conditions. An object-tracking algorithm that can effectively fuse high-frequency noisy inertial data (>1 kHz) with delayed, low-frequency (30 Hz) but accurate camera data has been designed. A practical system was constructed using cost-efficient components, and a dataset was collected for testing and characterization. The system demonstrated the ability to produce object pose estimates at 1 ms intervals. In experiments along a single axis, the system achieved a mean positional estimation error of 0.1 cm and a mean orientation estimation error of 0.5◦. The algorithm also successfully corrected for camera latencies of up to 350 ms while maintaining accurate estimates. These results demonstrate the possibility of achieving a low latency, accurate object tracking system while keeping component and computation costs reasonable.

This thesis offers a comprehensive exploration of log-based anomaly detection within the domain of cybersecurity incident response. The research describes a different approach and explores relevant log features for language model training, experimentation with different language models and training methodologies, and the investigation of the potential contribution of extra contextual features. The newly proposed approach is compared against an already implemented baseline in a finite-state classifier called FlexFringe, assessing their performance in detecting malicious anomalies across diverse datasets and hosts.

Key findings from this research underscore the importance of including human language for the generation of coherent clusters and a better performance of pretrained language models over models that were fine-tuned or built from scratch. Furthermore, the influence of clustering parameters on cluster quality proves to be crucial for cluster quality. Additionally, we gained insights into how extra contextual features are useful for log analysis.

In light of these findings, the study provides several recommendations for future research, including the expansion of the methodology to accommodate various log sources, the enhancement of preprocessing techniques, the integration of newer and more advanced language models, and the pursuit of efficient hyperparameter optimization. This work contributes to the continual advancement of log-based anomaly detection and its critical role in enhancing cybersecurity practices.

In recent years, there has been a growing interest among researchers in the explainability, fairness, and robustness of Computer Vision models. While studies have explored the usability of these models for end users, limited research has delved into the challenges and requirements faced by researchers investigating these requirements. This study addresses this gap through a mixed-method approach, involving 20 semi-structured interviews with researchers and a comprehensive literature analysis.
Through this investigation, we have identified a practical need for a data model that encompasses the essential information researchers require to enhance explainability, fairness, and robustness in Computer Vision applications. We developed a data model that holds the potential to improve transparency and reproducibility within this field, speed up the research process, and facilitate comprehensive evaluations, whether quantitative or qualitative, of proposed methodologies. To refine and demonstrate the practicality of the data model, we have populated it with four existing datasets. Additionally, we have conducted two user studies to validate the model's usability. We found that participants were enthusiastic about using the data model. Some potential uses described by the participants were comparing models and datasets, accessing (niche) datasets and models, creating and exploring datasets, and having access to ground truth explanations. However, participants also had concerns about the data model, mainly with its usability being restricted to people with database knowledge and the richness of data in the database. Nonetheless, hope that this research constitutes the first step for data modelling for researchers in the field of Trustworthy AI.

The advancement of wireless communication technologies has transformed how we exchange information in our daily lives. However, the increasing demand for wireless communication faces challenges due to limited radio wave bandwidth availability. In this context, visible light communication has emerged as a promising alternative, utilizing visible light for data transmission and reception. Screen-based visible light communication, also known as screen-camera communication, is a highly appealing option due to its exceptional user accessibility and seamless integration with existing infrastructure. It enables data to be embedded imperceptibly for humans within video content, ensuring a seamless and unobtrusive transmission.

Screens can be classified into two types: active screens, which emit their own light, and passive screens which harness ambient light as a more energy-efficient light source. Existing research has largely explored the use of active screens in screen-camera communication. This work focuses on the feasibility and implementation of a passive screen-camera communication link, addressing both data embedding within video scenes and decoding using an Android smartphone.

To accomplish this objective, an Android application is developed to decode data transmitted through a passive screen. Unlike most previous approaches that perform decoding in a non-real-time manner, this work implements a real-time application. However, because of the constraints imposed by the capturing rates of real-time cameras, both the transmission and reception are limited to low frame rates, specifically 15 frames per second transmission rate. It is anticipated that advancements in technology will eventually overcome this limitation. However, currently, the low transmission rate exhibits noticeable flickering. To reduce this issue, two novel encoding techniques were introduced, slightly reducing the flicker.

Through the exploration of a passive screen-camera communication link, this research further contributes to the advancement of visible light communication technologies. The proposed decoding techniques eliminate the reliance on visual screen reference points and address the challenge of real-time hand motion filtering, enhancing the practicality and usability of passive screen-camera communication systems in real-world scenarios. Finally, a proof-of-concept Android application was developed, effectively synchronizing audio playing on the smartphone with the video content displayed on a passive screen.

The response time performance of the passive screen-camera link was measured to be 530 ms when using a static phone holder and 1071 ms when handholding the phone.

The application of large language models (LLMs) for programming tasks, such as automatic code completion, has seen a significant upswing in recent years. However, due to their computational demands, they have to operate on servers. This both requires users to have a steady internet connection and raises potential privacy concerns. Therefore, this study aims to explore the feasibility of compressing LLMs for code using knowledge distillation (KD), thereby facilitating local usage of these models. Existing research has primarily focused on the efficacy of using KD to compress BERT models for language tasks. Its application to GPT models for coding tasks and the impact of implementing KD in-training, as opposed to the pre-training, remain largely unexplored. To address these gaps we adapted DistilBERT, a pre-training KD algorithm for distilling BERT models for language tasks. Our adapted model, Distil-CodeGPT, utilizes intraining KD to compress LLMs for Python code. The findings of this study suggest that a substantial reduction in model size is achievable, albeit accompanied by a compromise in predictive accuracy. Specifically, using 8 layers, instead of the original 12, resulted in a 24% reduction in disk size and a 28% speed increase, with an accompanying accuracy decrease of 11%. These results show that this approach has potential and is a solid first step toward smaller code models.

Code generation models have become more popular recently, due to the fact that they assist developers in writing code in a more productive manner. While these large models deliver impressive performance, they require significant computational resources and memory, making them difficult to deploy and expensive to train. Additionally, their large carbon footprint raises environmental concerns. To address these challenges, there is a need to develop techniques for compressing these models while maintaining their performance.
In this work, we study the effectiveness of Group lasso pruning and post-training quantization techniques on CPUs, applied to the code generation model CodeGPT. We evaluate the performance of the compressed model using the Exact Match (EM) and Edit Similarity (ES) metrics and study the model size on disk, memory footprint, and CPU inference. In contrast with the original CodeGPT model, our solution offers a 48% relative reduction in disk size, with only a mild drop in the accuracy metrics: 8.51% absolute drop in ES and a 5.5% in EM. Using the ONNX runtime on a regular laptop, we are able to deliver a 2x inference speedup at a 32.6% reduction in size. Our code is publicly available at https://github.com/AISE-TUDelft/LLM4CodeCompression/tree/main/CodeGPT-on-Intel.

The significant advancements in large language models have enabled their use in various applications, such as in code auto-completion. However, the deployment of such models often encounters challenges due to their large size and prohibitive running costs. In this research, we investigate the effectiveness of post-training quantization techniques in compressing a CodeGPT model, specifically using the "Per-embedding-group" and "Mixed precision" post-training quantization methods. Our evaluation is done on the code completion task of the CodeXGLUE benchmark using the Edit Similarity and Exact Match metrics, offering a comprehensive understanding of the impact of post-training quantization on the accuracy of the model. We also compare our results with three other compression approaches for the same model. From our analysis, we find that CodeGPT is very resilient to quantization noise, allowing the model to be compressed by 4 times its size with negligible accuracy loss. Furthermore, post-training quantization seems to be the best option for compressing the CodeGPT model when accuracy is a priority. Our work only simulates post-training quantization to draw conclusions on its performance on accuracy, future work should analyze the inference speed and memory use at runtime on such a post-trained quantized model.

Large language models are powerful because of their state-of-the-art language processing abilities. But, they come at the cost of being extremely resource-intensive, and are steadily growing in size. As a result, compressing such models for resource- constrained devices is an active and promising re- search area. In spite of their current popular- ity, many novel compression techniques lack im- plementation for GPT models. We apply the XTC pipeline, consisting of layer-reduction and quantisation through knowledge distillation, to a CodeGPT generative model. The resulting mod- els are evaluated on the CodeXGLUE line-level code-completion benchmark. Based on this, we demonstrate that (1) XTC can be adapted to GPT- like models, translating many of the findings of the original study; (2) a 6-layer reduction with 1-bit weight and 8-bit activation quantisation is able to reduce model size 15×, in addition to almost dou- bling inference speed, with minimal performance degradation. The resulting compressed models show promise for use in local code generation. By showing that a novel compression technique can be adapted to GPT-like models, we hope to further in- spire research in this field.

The main principle of Open Source development is that developers can reuse different libraries over and over again to make their lives easier. That is why this practice has gained a lot of popularity. However, libraries usually depend on other already existing pieces of code. This means that whenever some small piece of code fails, the whole application may crash due to its dependencies. Since Debian is considered as one of the largest community run distributions, it is important to have a good tool to analyse these dependencies to help in avoiding such crashes. That is why this research will focus on building a dependency graph for Debian's package manager and finding out which are the most widely used packages. What separates this research from previous related works, is the addition of a time component to the graph. This will be in the form of a timestamp of the release date for each version of a package. This allows for extensive traversal of the graph, which can also be based on time periods. By doing so, no transitive dependencies should be missed. The paper concludes with defining most widely used as the packages which achieve the highest PageRank score when put into the graph. The top 3 ones are "libc6", "libgcc1" and "multiarch-support".

In (open-source) development, developers routinely rely on other libraries to improve their coding efficiency by reusing code. This reliance on other packages could cause issues when critical dependencies have suddenly have a vulnerability introduced to them. This work analyzes the criticality for NPM. To get an accurate picture of what the most-critical and thus possibly most-vulnerable packages are, the entirety of NPM must be analyzed. However, this proved too big to be able to fit in 500GB of memory. This work therefore examines a small subset of 100 thousand packages. To do the analysis, this paper proposes a novel approach of embedding a time dimension into the package network to provide better accuracy. This papers analysis show that both with and without this time dimension, \texttt{babel} packages are by far the most important in the package graph (as measured by PageRank). We should, however, keep in mind that this came from only analyzing 100 thousand packages. Thus, further research is required to confirm this conclusion. In particular, other importance measures should be used to find out the packages' criticality.

Developers rely on different software to improve their efficiency as to reuse parts of code and be able to maintain it with ease, which is why open source software libraries have gained much pop- ularity over the past years. This paper analyzes what are the most used packages from Apache Maven, which is a build automation tool used primarily for Java projects. In order to do an accurate analysis, it is necessary to collect all the packages and their dependencies from the Maven Central Repository. But, as of 5th of May 2022, analyzing the whole repository proves to be an extreme challenge, as the repository has 9M of indexed artifacts, which contain metadata such as the version and a list of dependencies. Besides, there are many more other repositories that also contain such artifacts. Thus, this pa- per examines only a small subset of artifacts and provides a data structure that is able to take into consideration the time dependency between li- braries, which can be also scaled to suit a much bigger input. The paper concludes with the most used software on given time-frames, but with- out specifying any time frames, the popular junit package comes out first.

The use of open-source packages is a common practice among developers. It decreases the development time and improves maintainability. But adding a dependency to a project comes with inherit risks such as introducing vulnerabilities. A few solutions that help visualize all of the dependencies of a project exist already. However, none provide the capability of selecting a moment in time for analyzing the generated structure. This research paper formalizes a time-based dependency graph that can be generalized to any ecosystem and then showcases its usefulness by analyzing the Python ecosystem throughout time. The results indicate that the Python ecosystem does have a subset of packages that are the most used - such as numpy and requests - but overall it is well balanced, meaning that it is able to withstand the removal of one of its most used packages. The data structure also provides satisfactory results, having a 89.6% accuracy when compared to the Python resolver. The findings of this study can be used to improve existing dependency networks.

Deficiencies in online communication software inspired the HoloLearn project that proposes to aid hybrid education via a hologram. Yet the epistemic and psychological effects of this medium remain largely uncharted. To help integrate the hologram into a hybrid education system, the present study conducted experiments which explored the nature of the hologram itself. With a lecture on Japanese History until 1603, this research compared the hologram's twofold effects on students with Zoom, an already popular option for online lecturing. The study concludes that there is no statistically significant difference between the two teaching modalities, and also suggests that the hologram, depending on the viewer, can be either engaging or distracting. Flaws in experimental design and other external constraints might have exerted some influence on these findings.

This study examines the effects of holograms when used for online learning. The research question is: "How does a holographic teacher projection affect engagement of the students with the learning material?". The engagement of students was compared in different lectures. The base lecture was a zoom lecture. The holographic lectures were a VR lecture, a HoloDisplay lecture and a robot lecture. A big experiment was done where different people attended one of those 4 lectures. After the participants were done watching the lectures they had to fill in questionnaires. The data of these questionnaires was then used to compare the lectures. The main conclusions are that Zoom lectures definitely take more attention than holographic lectures. Another conclusion was that the HoloDisplay and the Robot lecture are more engaging than a normal zoom lecture, this conclusion was backed up with statistical tests. There was no evidence that either the HoloDisplay or the Robot lecture was more engaging than the other one. Both of them need to be looked into further and be developed further. These ways of teaching can definitely be the future of online learning.

Distance learning brings all sorts of advantages. The ability to follow lectures at home can save people transportation costs and time. Teaching through videoconferencing software such as Zoom is one of the methods to learn remotely. To explore new and better teaching methods, the possibility of using a telepresence robot is investigated. To determine its worth, an experiment is done to compare the learning outcomes of telepresence robot lectures to Zoom lectures. Differences in risk of eye-strain is also compared, as most distance learning methods are digital. An increase in risk of eye-strain may negatively impact the choice of adopting the learning method. Due to the small sample size, non-parametric tests were conducted. Concluding from the research, neither Zoom nor telepresence robot lectures had better learning outcomes compared to the other. The learning outcomes of both lectures were similar. Risks of eye-strain also did not differ between the two. Further analysis was done comparing the telepresence robot lecture with a hologram and a virtual reality lecture, as these were also part of the experiment. Similarly to zoom, there was no significant evidence that telepresence robot lectures had different learning outcomes or risks of eye-strain than holograms and virtual reality lectures.