Building predictive models using cloud metrics for a task like incident prediction in the cloud is becoming ubiquitous in cloud monitoring. For such a forecasting task, if we know beforehand which system metrics are predictable then we can easily build good models. Quantifying the predictability of cloud metrics can help us rank the available system metrics and select a subset of cloud metrics with the lowest complexity. Moreover, storing informative metrics for a longer period can result in better forecasting. This thesis presents a novel entropy method for quantifying the complexity of time series: Reverse weighted Dispersion Entropy (RWDE). We also present an exploratory study to understand and quantify the complexity of cloud metrics. This exploratory case study has been carried out at ING, a large banking company with in-house cloud architecture. We perform simulation experiments on simulated signals to compare RWDE with other entropy methods. We apply RWDE on cloud metric data from ING to approximate the predictability of these cloud metrics. The experimental results show that RWDE has better performance than other entropy methods and can be used to select informative cloud metrics for a forecasting task. Further, we establish a relationship between RWDE and model-based predictability of cloud metrics. For each cloud metric, we compare RWDE with predictions from various forecasting models. Our results show that this relationship can be used as a heuristic by practitioners to identify unsuitable forecasting models for certain cloud metrics. We make RWDE and other entropy methods discussed in this study available as an open-source Python package.

Abstract interpretation is a way of approximating the semantics of a computer program, in which we derive properties of those programs without actually performing the necessary computations for running the program, through the use of an abstract interpreter. To be able to trust the result of the abstract interpretation, we would to able to prove the soundness of the approximations of the interpreter. Previous work by Keidel et al. has shown that the soundness proofs of an entire abstract interpreter can be simplified by decomposing the interpreter by implementing concrete and abstract interpreters as instantiations of a generic interpreter. The goal of this thesis is to explore and implement mechanical proofs of soundness of such interpreters. To this end, we have used the interactive proof assistant Coq to implement a generic interpeter for a simple imperative language and instantiate it both concrete and abstract versions. The abstract interpreter is automatically proven sound via the use of Coq's automatic proof capabilities and typeclass system. Both the interpreted language and the used abstractions can be expanded to allow for more features. Soundness proofs can then be written for just the new components, those proofs will then be automatically resolved by Coq.

Cloud datacenters provide a backbone to our digital society. Crucial to meeting increasing demand while maintaining efficient operation is the activity of capacity planning. Inaccurate capacity planning for cloud datacenters can lead to significant performance degradation, denser targets for failure, and unsustainable energy consumption. Although this activity is core to improving cloud infrastructure, relatively few comprehensive approaches and support tools exist, leaving many planners with merely rule-of-thumb judgement.

We propose Capelin, a data-driven, scenario-based capacity planning system for cloud datacenters. We design Capelin to address requirements we have derived from a unique survey of experts in charge of diverse datacenters in several countries. Capelin introduces the notion of portfolios of scenarios, which it leverages in its probing for alternative capacity-plans. At the core of the system, a trace-based, discrete-event simulator enables the exploration of different possible topologies, with support for scaling the volume, variety, and velocity of resources, and for horizontal (scale-out) and vertical (scale-up) scaling. The approach centers around a notion of portfolios of scenarios as a framework for probing alternative decisions and courses of events. Capelin gives detailed quantitative operational information for each scenario, which could facilitate human decisions in capacity planning.

We implement and open-source Capelin, and show through comprehensive trace-based experiments it can aid practitioners. Although Capelin is designed to work across many kinds of datacenters, in this work we focus on private-cloud, business-critical workloads, and on public-cloud operations. The results give evidence that choices that seem reasonable and common in practice could be worse by a factor of 1.5-2.0 than the best, in terms of performance degradation or energy consumption. We also show evidence of Capelin identifying meaningful choices that are different from the baseline proposed by a team of professional datacenter engineers. We open-source Capelin and release data artifacts for public inspection and reuse.

The amount of data being collected is growing exponentially, both in academics as well as in business. Unfortunately, the quality of that data can be poor, leading to poor decisions and increasing costs. Data cleaning, the process of detecting and correcting errors from a dataset, could be the solution to improve bad data.
This research focuses on detecting these errors. There are (semi-)automated error detection tools available, but it is unclear how well these tools perform under varying conditions and on different datasets.
Following from this problem, the main research question was developed: How to
choose a fitting error detection algorithm for a specific relational dataset?
To answer this question, a comparative study has been done for error detection tools on relational data. An interactive error detection tool, Raha, performed best from the selected state of the art tools.
Subsequently, an attempt was made to estimate the performance of error detection tools and particular configurations on unseen datasets, based on high-level profiles of these datasets. According to the qualitative and quantitative experiments in this research, the proposed estimators have been shown to be effective. Moreover, the performance estimators were analyzed to provide more interpretability on the functioning of the error detection tools on the datasets in this research.
Ultimately, these performance estimators were used to generate suggested rankings of error detection strategies. The produced system outperformed the set baseline and was able to create valuable rankings. The proposed strategy ranking system could help real-world computer scientists and data experts choose a fitting error detection algorithm for a specific relational dataset.

The recommendation of tasks for newcomers within a software project throughgood first issues is being done within the domain of software development, such as onGithubplatform. These issues aim to help newcomers identify tasks that are suitablefor them and their level of expertise within the project. This thesis report investigatesthe effectiveness regarding developer onboarding and task completion ofgood firstissues by data mining a set of 105 repositories and manually analyzing at most 30good first issues and 30 initial commits per sampled project. It was found that, althoughgood first issues are effective at developer onboarding, and developers perceivegoodfirst issues as being useful, changes can be made to the types of tasks suggested asgood first issues to match the types of initial contributions made by newcomers. It wasalso found that developers with less than a year of experience favoreddocumentation-related contributions for their first commit to a project.

Systematic literature reviews in software engineering as well as other disciplines, serve as the foundation for sound scientific research. The aim for these literature reviews is to aggregate all existing knowledge on a research problem and produce informed guidelines for practitioners. This enables practitioners to apply appropriate software engineering solutions in a specific contexts. However, one major problem exists regarding systematic literature reviews, the overall execution duration may take up as much as 24 months. The first objective of this study is to provide a solid base for the AI for FinTech Research collaboration by performing a systematic literature review. This literature review is used to identify different machine learning techniques in the context of the FinTech domain. However, during this study, we found that a significant amount of time was spent on repetitive work which potentially could have been automated. Therefore, the second objective of this work is to reduce the overall workload for performing systematic literature reviews. First, a literature review is performed regarding automation solutions for different steps of systematic literature reviews. The identified solutions were used to create a tool to automate steps in both the retrieval and screening phase of systematic literature reviews. First, this work presented the state of the art regarding machine learning applications in the FinTech domain. Afterwards, a complete overview of possible automation solutions for every step of performing literature reviews was detailed. Using this overview, a tool was created which showed that the overall workload of the retrieval and screening phase of systematic literature reviews can be significantly reduced.

Since building a machine learning model costs a lot while following 9 stages, the automated machine learning model creation became a crucial role in a large-scale context. At the same time, a monitoring system became an essential factor for machine learning models. This thesis presents the monitoring system for machine learning models at ING in an enterprise context with new features required by users. Moreover, the thesis describes a case study of ING, a large global banking company that develops software solutions in-house. We conducted a mixed-methods study, consisting of data collection of the monitoring system and a survey with the users of the monitoring system. Our research shows that challenges found by the actual users of the monitoring system and mapped challenges discovered by the Microsoft study are related to machine learning model monitoring, the perception of the users on the importance of the monitoring system, and the impact of the monitoring system. We found that the monitoring system at ING supports relatively efficient model management in terms of checking model validation and evaluation. Moreover, the users of the monitoring system perceived that it is an important system, and it supports the models regarding quality, the trust of the automated model creation, and usability. Additionally, compared to the existing solution, the monitoring system at ING supports useful model management.

While the amount of data and variability in data produced by numerous systems in a modern company continues to increase, users desire real-time and consistent results from complex analyses across a large variety of event sources. In industry, stream processing systems are emerging to process events with low latency in a scalable and reliable fashion. As more and more stream processing jobs are processing mission critical events, older jobs are subject to maintenance and have to be upgraded or replaced. These upgrade operations include a snapshot-restore operation, where between the snapshot and restore a non-trivial state conversion has to be performed. Such an operation requires a lot of technical expertise and imposes significant down-time on the job itself and all jobs that depend on it. This thesis proposes a mechanism to align the progress of multiple independent jobs sharing common event sources. The mechanism is an extension of the checkpoint protocol proposed by Carbone et al. Not only does this mechanism simplify maintenance of streaming jobs by allowing hot-swap operations with exactly-once processing semantics, but it can also be used to provide consistency of queryable state. By implementing a proof of concept we show that this so called epoch alignment can be achieved with minimal additional costs over exactly-once processing semantics.

CodeFeedr is a research project at the software engineering division of the Delft University of Technology in collaboration with the Software Improvement Group. The research focuses on a software infrastructure which serves software practitioners in utilizing data-driven decision making. Currently, frameworks like Apache Flink are capable of high-performance data streaming. However, these frameworks have a lot of overhead in setting up, and adding new streaming queries takes a lot of time. They also have several limitations in combining real-time data with historical data and doing aggregations on streams from multiple sources. The developed product is a plug-in framework on top of Apache Flink, that provides a pipelining system for streaming queries. This product includes abstractions for well-known sources like GitHub, TravisCI and Twitter as well as support for historical data in mongoDB. With this framework the users can spend their efforts on actually writing streaming queries instead of setting up environments, input sources and output destinations. The product also includes orchestration tools for running streaming jobs on a distributed system.

Code review has been proved to be an extremely important practice to ensure softwarequality. In recent years, the trend has moved towards modern code review, a lightweight and less strict paradigm. Despite its many advantages, this approach still has room for further improvement, especially in the area of cognitive support for reviewers.
Previous research stated how ordering code changes based on their relations may constitute an effective way to support reviewers. Based on this premise, this work focuses on studying how this ordering theory may be applied in practice. As result, a tool that automatically orders the modifications in a commit has been created.
Moreover, the tool has been tested and an initial investigation of the perceived
usefulness of its relations has been conducted. Finally, it has been investigated if the ordering produced by the tool is identified as useful by the developers and which factors may influence this choice.

Debuggers are crucial tools for developers to support the process of developing software systems as they provide direct insights into the execution of their code. As software development in the industry is moving towards technology stacks that operate on increasingly higher levels of abstraction, debugging tools have not evolved as quickly. This creates an abstraction gap between the concrete debugging needs of developers and the support that debugging tools offer. To reduce this gap, we propose a generic framework for reactive scriptable debugging. We propose a design and reference implementation of such a scriptable debugging system. Using this framework we aim to liberate developers from rigid debugging tools and give the power of debugging back to the developers.

Rabobank is currently planning a complete transition of its services to the public
cloud. Currently there are around 400 DevOps teams that need to make the
transition from deployment on traditional on-premise infrastructure, to deploying their products to the public cloud. The thesis project investigates what are some of the biggest challenges in adopting cloud computing technologies in an enterprise.
Some of these are encouraging the staff to adopt the technology, how to
embed security and compliance to the cloud computing infrastructure and which
services to choose when migrating an on-premise application to Microsoft Azure Cloud. Investigation has been done on how to better encourage and inform the staff about the cloud adoption. This is achieved by improving the Cloud Awareness session (where the employees are informed for the cloud initiatives in the bank) through implementation of complete CI/CD (Continuous Integration / Continuous Deployment) pipeline of a .NET Core 2.0 application with modern HTML5 responsive layout that deploys the web application on Microsoft Azure Public Cloud and gives recommendations for testing and monitoring. Moreover, the project investigates what is needed to develop the secure and compliant feature in a huge enterprise like Rabobank with an example of development an Azure Cosmos DB feature delivered as a VSTS (Visual Studio Team Services) extension to be used by the DevOps teams in their CI/CD pipeline. Finally, an analysis is done on the performance, cost and lifecycle management of the same .NET Core 2.0 application deployed on different service offerings by Microsoft Azure Public Cloud, including Windows Server virtual machine, Azure Web App Service and Azure Kubernetes Service.

The continuous shift of various industries towards internet-based services have caused an exponential growth in the amount of data produced over the past few years. On top of this, the increasing need for real-time analytics and the increase in data velocity have made asynchronous, event-driven applications the norm. In this context, the reactive programming paradigm has gained much traction as it focuses on the propagation of change and composing/transforming streams of data. The industry standard reactive programming library for the JVM, .NET and Javascript ecosystems is the Reactive Extensions (Rx) library.
However, despite being well equipped to deal with asynchronous data, it does not offer any way of scaling the computation on multiple machines. In this thesis, we attempt to lay the groundwork for a scalable Rx library by implementing infrastructure and operators for remote execution of Rx streams.

Software testing has been around for decades and many tools exist to aid developers in their testing process. However, little is known about the rate at which developers test their projects, the tools they use for these purposes and the impact of type systems on testing practices. Our work is the first of its kind to identify and classify available test tooling for programming languages Java, C, JavaScript and Ruby. By conducting a large scale automated analysis on open-source software projects, we show that both available test tooling and type systems have significant impact on codified testing strategies. Our observations are strengthened by targeted interviews and a large-scale survey among developers working with both statically and dynamically typed programming languages. The soft typing system seems like a promising solution, allowing developers to work with the great flexibility less strict type systems provide, while simultaneously benefiting from strict type checks that help reduce the amount of testing required to ensure the correctness of a piece of software. However, future research is needed to estimate the feasibility of such a type system. To help better aid developers in their testing process and to reduce some of the testing burden many developers seem to cope with, we propose a list of test tooling improvements based on our observations.

A linter is a type of static analysis tool that warns software developers about pos- sible errors in code or violations to coding standards. By using such a tool, errors can be surfaced early in the development process when they are cheaper to fix, and code can be kept more readable and maintainable. For such a tool to be successful, it is important for its creators to understand the needs and challenges of developers when using a linter. Furthermore, it needs to be made clear to developers why using such a tool can be beneficial, along with how linters can be configured to identify appropriate and relevant issues for their projects.
In this thesis, we examine developers’ perceptions of linters to increase our knowl- edge on these tools for JavaScript, the most widely used programming language in the world today. More specifically, we study why and how developers use ESLint, the most popular JavaScript linter, along with the challenges that they face while using the tool. We collect data with three different methods where we first interview 15 experts on using linters, then analyze over 9,500 ESLint configuration files and finally survey more than 300 developers from the JavaScript community. The combined results from these analyses provide developers, tool makers and researchers with valuable knowl- edge and advice on using and developing a linter for JavaScript.