A Tale of CI Build Failures

An Open Source and a Financial Organization Perspective

Conference paper (2017)

Authors

Carmine Vassallo University of Zürich
Gerald Schermann University of Zürich
Fiorelli Zampetti University of Sannio
D. Romano ING
Philipp Leitner University of Zürich
A.E. Zaidman Software Engineering - EEMCS
Massimiliano Di Penta University of Sannio
Sebastiano Panichella University of Zürich
Research Group
Software Engineering (EEMCS) (TU Delft)
Copyright: © 2017 Carmine Vassallo, Gerald Schermann, Fiorella Zampetti, D. Romano, Philipp Leitner, A.E. Zaidman, Massimiliano Di Penta, Sebastiano Panichella
DOI: https://doi.org/10.1109/ICSME.2017.67
More Info
expand_more

Published Date

2017

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Software Engineering

Abstract

Continuous Integration (CI) and Continuous Delivery (CD) are widespread in both industrial and open-source software (OSS) projects. Recent research characterized build failures in CI and identified factors potentially correlated to them. However, most observations and findings of previous work are exclusively based on OSS projects or data from a single industrial organization. This paper provides a first attempt to compare the CI processes and occurrences of build failures in 349 Java OSS projects and 418 projects from a financial organization, ING Nederland. Through the analysis of 34,182 failing builds (26% of the total number of observed builds), we derived a taxonomy of failures that affect the observed CI processes. Using cluster analysis, we observed that in some cases OSS and ING projects share similar build failure patterns (e.g., few compilation failures as compared to frequent testing failures), while in other cases completely different patterns emerge. In short, we explain how OSS and ING CI processes exhibit commonalities, yet are substantially different in their design and in the failures they report.