Technical debt is a term that describes the consequences of taking shortcuts or quick-and-dirty solutions in the software engineering process, in order to gain short term advantages in the development process of software projects. In this paper, we investigate the technical debt present at the fintech company Adyen, and the developers' experience with technical debt. We investigate their behavior towards testing and refactoring, and give insight into module health with the Delta Maintainability Model. With the conclusions from this research, we propose a novel gamification system called 'Code Karma' to address these technical debt issues, give insight into efforts that increase code quality and motivate developers to improve internal software quality. We found that the Delta Maintainability Model may be used to evaluate module health over time. Additionally, developers believe that 'Code Karma' has a positive influence on the internal quality of the system.

Practice is central in mathematics skill acquisition. The practice process can be facilitated by flexible digital exercise systems, supporting personalized learning and providing students with parameterized, open answer exercises containing answer-specific feedback. However, current solutions for authoring these exercises lack efficiency and expressiveness, or rely on unpopular languages for content generation. To address these issues, this work demonstrates a flexible setup for authoring exercises in Jupyter Notebook cells using Python and the available libraries therein and evaluates this setup with expert users and authors of exercises.

Active state machine learning algorithms are a class of algorithms that allow us to infer state machines representing certain systems. These algorithms interact with a system and build a hypothesis of what the state machine describing that system looks like according to the behavior they observed. Once the algorithm arrives at a hypothesis, it sends it to a so-called equivalence oracle to be checked. The equivalence oracle returns a counterexample trace describing different behavior between the hypothesis and the real system if one exists. Traditionally, this task is done using a technique called the W-method, which provides certain guarantees about finding a counterexample if one exists, but this is rather slow due to the thorough search it needs to do. In real systems, there are sometimes patterns to the counterexamples to be found during the learning process, which can be exploited to speed up the process of finding new counterexamples. Instead of performing an exhaustive search each time, we first use the already observed patterns to generate new traces to try before falling back to traditional techniques in case no counterexample was found in the first step. This allows us to find counterexamples more quickly than usual, and because we can always fall back on the techniques that do provide guarantees we do not have to sacrifice correctness. In the same amount of time, our method provides an up to 26x increase in the number of states found compared to the W-method, while remaining fully black box and without degrading worst-case performance. We also show that, when not working in a fully black-box scenario, fuzzing can be a successful complementary technique to apply next to active learning. However, we also show that relying on fuzzing alone does not always fully capture the behavior of a system, and when correctness is critical such as in the case of model checking, it needs to be supplemented with additional techniques to result in fully correct models. Finally, we use our home-grown state machine learning library to participate in the 2020 RERS challenge, and apply the techniques we studied so far to attain a perfect score on the sequential LTL track.

Attribution of the malware to the developers writing the malware is an important factor in cybercrime investigative work. Clustering together not only malware of the same family, but also inter-family malware relations together provides more information about the authors and aid further malware analysis work. In this report, previous work which concluded attribution on compiled binaries can be done by a programmer their style is questioned. Given insight on this matter, this report explores new clustering techniques for both static and dynamically derived features from malware binaries. Both methods are complementary as they provide very different types of data. In the static analysis, the data for the similarity comparison is derived from disassembled binaries, while in dynamic analysis the choice was made to record system calls executed by the malware during ecution.
We use a finer granularity than when comparing the data of the complete binaries with each other, such that instead of differences, fine similarities among malware families can be found. Evaluation of clusters is a difficult subject, because of its unsupervised nature and data quality related causes. However, upon manual inspection of the generated clusters, the newly developed clustering methods confirm previously discovered similarities but also find new connections among malware families.

Modern web information systems use machine learning models to provide personalized user services and experiences. However, machine learning models require annotated data for training, and creating annotated data is done through crowdsourcing tasks. The content used in annotation crowdsourcing tasks like medical records and images might contain some private information which can directly or indirectly identify an individual. The name, age, ethnicity, gender, contact details are examples of private information that directly identifies an individual. Indirect private information relates to the cultural, economic, and social factors of an individual. For instance, the visual cues of religious objects or symbols relate to the religious beliefs of an individual. In this thesis, we study how to minimize the amount of private information extracted from images using a hybrid algorithm which combines machine learning models and crowdsourcing. We also demonstrate that the proposed hybrid algorithm reduces the amount of private information exposed from the image and the cost of using the crowd for detecting private information in the image.

Huge amounts of log data are generated every day by software. These data contain valuable information about the behavior and the health of the system, which is rarely exploited, because of their volume and unstructured nature. Manually going through log files is a time-consuming and labor-intensive procedure for developers. Nonetheless logging information can expose the problematic execution of the software, even though the final outcome seem to be normal. Nowadays the automatic analysis of the log files is crucial for detecting problems, but mainly for understanding how the software behaves, which would be beneficial for the prevention of failures and improvement of the software itself. Towards that direction, this project aims the identifications of unexpected executions of the software and the determination of the root cause behind them. In more details, the expected behavior of the software can be approximated using model inference techniques and the newly incoming observed data can be analyzed to verify if they are conformed by the expected behavior. The conformance checking method that will be used is called replay. The incoming traces will be replayed in the graph, at the point they are not validated, the alignment algorithm will take over. The sequence alignment is performed in three different ways. Two of the methods are looking for the best alignment at a specific radius around the problematic node. Additionally a global alignment technique is implemented, which is based on the famous algorithm by Needleman and Wunsch for DNA sequences. Our goal required the modification of the aforementioned algorithm to not only align two sequences, but a sequence with a tree structured model. Finally the implemented tool provides the visualization of the differences in a way that makes it intuitive for the developers to understand what went wrong. Some additional information are also provided to make the investigation of the "anomaly" easier.

Code reviews are a widely adopted practice in software engineering that is proven to increase the quality of the code. Despite its evolution in the last decade, it still presents a number of challenges, such as understanding the changeset in review. In this thesis we research the usage of Software Visualisation paradigms to aid reviewers in the change-understanding process with a tool-based approach. Based on a survey of the code change visualisation and code navigation research areas, we devise a set of candidate prototypes of a cognitive support review tool, which we iteratively refine involving developers in the process. Through an online survey, we select one of them and build CHANGEVIZ, the implementation of our novel code review environment. The effectiveness of our approach is validated with a preliminary experiment in which developers perform change-review tasks in our review environment. The results suggest that the features incorporated by our tool are valuable for reviewers.

The rail network in the Netherlands is one of the busiest in Europe. To ensure a safe and reliable infrastructure, preventive maintenance is of utmost importance. ProRail, the sole maintainer of the railway infrastructure in the Netherlands, spends hundreds of millions of dollars on maintenance each year. Due to the complexity and busyness of the network, maintenance can cause major disruptions leading to longer travel times. Despite these factors, maintenance is currently scheduled mostly manually, leading to suboptimal schedules being created. A pilot study by Macomi, the company which enabled this study through an internship, showed great potential for improvement in scheduling this maintenance. In this thesis, the aim is to further improve the maintenance schedule of ProRail, as well as to find out what kind of solution method is most suitable to improve that schedule. To achieve this goal, various types of algorithms have been implemented and tested on the problem. An evolution strategy algorithm developed by Macomi was successfully improved. Through the use of multi-objective algorithms, the trade-off between maintenance costs and availability of the infrastructure was analyzed. These multi-objective algorithms were found to provide unsatisfactory solutions. Greedy algorithms were also developed to provide a quicker solution method, and the resulting solutions were of surprisingly high quality. Finally, a hybrid algorithm using the evolution strategy and a greedy algorithm was created. It was shown that this hybrid algorithm outperforms all other algorithms, and provides solutions that are better in terms of costs and constraints compared to the manual schedule of ProRail, and the baseline established by the pilot study of Macomi.

Software development is more than only implementing the functional code. A developer is also responsible for writing code measuring up to certain standards and conventions. These conventions make sure that the code is of a particular quality that improves readability and eases maintainability. Some of these conventions are checked by automated tools. Automated static analysis tools (ASATs) perform an analysis of the source code and issue warnings. ASATs are available for many programming languages and can be used to find functional or maintainability issues. Even though these tools have been proven to be useful during the code development process, developers do not always utilize them. The overload of warnings in large projects and relatively low importance of these warnings are one of the many reasons why they are ignored. In this study, a gamification tool, Checkpoint, is developed based on the GOAL methodology. The purpose of this tool is to gamify the development process pertaining to ASATs to motivate developers. The developers are motivated using various gamification elements during a pretest-posttest pre-experimental experiment. The study tested the usability of the tool and its effectiveness. The experiment showed that gamification has an impact on developer motivation.

When a new software developer joins bunq, a Dutch bank, (s)he will need some time to get familiar with the existing codebase. Preferably, the time necessary for the familiarisation is as short as possible. The faster the developer is up to speed, the faster (s)he can contribute to new features and help solving bugs. This research develops a prototype for a tool that aims to support developers in the comprehension of the PHP backend of bunq. This tool has been evaluated by interviewing multiple developers. Furthermorewe asked the developerswhether the toolwould be able to replace a senior developer regarding question asking. During the interviews feedback was acquired on the current prototype. New developers found the tool useful, although there was also room for improvement. More experienced developers indicated that they found the tool less useful, but saw opportunities to use the tool for more managerial-like tasks.