E-Compare: Automated energy regression testing for software applications

Abstract

As data centers worldwide consume more power than ever, lowering the energy consumption of software is increasingly important. Software energy testing is often unclear due to a lack of comparable baselines. In this paper, we look at the use of regression testing to alleviate some of the struggles with energy testing. We introduce E-Compare, a tool designed to identify energy regressions in software updates by comparing the energy consumption of different versions of the same project. E-Compare is cloud-based, fully automated, and can be implemented in any project with just three lines of code. To validate its effectiveness, we applied E-Compare to thirteen real-world projects, ranging from long-established projects to newer, active ones. Over 700 code changes have been tested. Our findings indicate that energy regression testing can identify energy regressions missed by developers. Some of the indicated energy regressions could be traced back to specific code changes, confirming the tool’s accuracy and relevance. However, the tool’s usability varies significantly depending on the project, and unexpected energy regressions are relatively rare.