The principles of Findability, Accessibility, Interoperability, and Reusability (FAIR) have been put forward to guide optimal sharing of data. The potential for industrial and social innovation is vast. Domain-specific metadata standards are crucial in this context, but are widely missing in the energy sector. This report provides a collaborative response from the low carbon energy research community for addressing the necessity of advancing FAIR metadata standards. We review and test existing metadata practices in the domain based on a series of community workshops. We reflect the perspectives of energy data stakeholders. The outcome is reported in terms of challenges and elicits recommendations for advancing FAIR metadata standards in the energy domain across a broad spectrum of stakeholders.

@en

Research data management (RDM) is increasingly important in scholarship. Many researchers are, however, unaware of the benefits of good RDM and unsure about the practical steps they can take to improve their RDM practices. Delft University of Technology (TU Delft) addresses this cultural barrier by appointing Data Stewards at every faculty. By providing expert advice and increasing awareness, the Data Stewardship project focuses on incremental improvements in current data and software management and sharing practices. This cultural change is accelerated by the Data Champions who share best practices in data management with their peers. The Data Stewards and Data Champions build a community that allows a discipline-specific approach to RDM. Nevertheless, cultural change also requires appropriate rewards and incentives. While local initiatives are important, and we discuss several examples in this paper, systemic changes to the academic rewards system are needed. This will require collaborative efforts of a broad coalition of stakeholders and we will mention several such initiatives. This article demonstrates that community building is essential in changing the code and data management culture at TU Delft.

@en

In this paper, we explain our strategy for developing research data management policies at TU Delft. Policies can be important drivers for research institutions in the implementation of good data management practices. As Rans and Jones note (Rans and Jones 2013), " Policies provide clarity of purpose and may help in the framing of roles, responsibilities and requisite actions. They also legitimise making the case for investment”. However, policy development often tends to place the researchers in a passive position, while they are the ones managing research data on a daily basis. Therefore, at TU Delft, we have taken an alternative approach: a policy needs to go hand in hand with practice. The policy development was initiated by the Research Data Services at TU Delft Library, but as the process continued, other stakeholders, such as legal and IT departments, got involved. Finally, the faculty-based Data Stewards have played a key role in leading the consultations with the research community that led to the development of the faculty-specific policies. This allows for disciplinary differences to be reflected in the policies and to create a closer connection between policies and day-to-day research practice. Our primary intention was to keep researchers and research practices at the centre of our strategy for data management. We did not want to introduce and mandate requirements before adequate infrastructure and professional support were available to our research community and before our researchers were themselves willing to discuss formalisation of data management practices. This paper describes the key steps taken and the most important decisions made during the development of RDM policies at TU Delft.@en

The evolution of highly configurable systems is known to be a challenging task. Thorough understanding of configuration options their relationships, and their implementation in various types of artefacts (variability model, mapping, and implementation) is required to avoid compilation errors, invalid products, or dead code. Recent studies focusing on co-evolution of artefacts detailed feature-oriented change scenarios, describing how related artefacts might change over time. However, relying on manual analysis of commits, such work do not provide the means to obtain quantitative information on the frequency of described scenarios nor information on the exhaustiveness of the presented scenarios for the evolution of a large scale system. In this work, we propose FEVER and its instantiation for the Linux kernel. FEVER extracts detailed information on changes in variability models (KConfig files), assets (pre-processor based C code), and mappings (Makefiles). We apply this methodology to the Linux kernel and build a dataset comprised of 15 releases of the kernel history. We performed an evaluation of the FEVER approach by manually inspecting the data and compared it with commits in the system’s history. The evaluation shows that FEVER accurately captures feature related changes for more than 85% of the 810 manually inspected commits. We use the collected data to reflect on occurrences of co-evolution in practice. Our analysis shows that complex co-evolution scenarios occur in every studied release but are not among the most frequent change scenarios, as they only occur for 8 to 13% of the evolving features. Moreover, only a minority of developers working on a given release will make changes to all artefacts related to a feature (between 10% and 13% of authors). While our conclusions are derived from observations on the evolution of the Linux kernel, we believe that they may have implications for tool developers as well as guide further research in the field of co-evolution of artefacts.@en

Most modern software systems can be adjusted to satisfy sets of conflicting requirements issued by different groups of users, based on their intended usage or execution context. For systems where configurations are a core concern, specific implementation mechanisms are put in place to allow the instantiation of sets of tailored components. Among those, we find selection processes for code artefacts, variability-related annotation in the code, variability models representing the available features and their allowed combina- tions. In such systems, features, or units of variability, are scattered across the aforementioned types of artefacts. Maintenance and enhancement of existing systems remain a challenge today, for all types of software systems. But in the case of variant-rich systems, engineers face an additional challenge due to the complexity of the product instantiation mechanisms: the maintenance of the variability model of the system, the complex build mechanisms, and fine-grained variability in the source code. The evolution of the system should be performed such that the information contained within the various artefacts remains consistent. In practice, this means that as the implementation of the system evolves, so should the mechanisms put in place to generate tailored products. Little information is available regarding changes occurring in such systems. To efficiently support such developers tasks and ease maintenance and enhancements activities, we need a deep understanding of the changes that take place in such systems. The state of the art provides trends over long period of times, highlighting systems growth - such as number of added or removed features in each release, or the evolution of cross- tree constraints in a variability model. While important to describe the core dynamics behind the evolution of a system, this does not provide information on the changes per- formed by developers leading to such trends. Similarly, this global information cannot be leveraged to facilitate developers’ activities. The focus of this thesis is the acquisition and usage of change information regard- ing variant-rich system evolution. We show how the information lacking from today’s state-of-the-art can be obtained from variant-rich system change history. We propose a set of tool-supported approaches designed to gather such information and show how we leverage change information for change impact analysis, or to derive knowledge on developer practices and the challenges they face during such operations. With this work, we shed new light on change scenarios involving heterogeneous artefacts regarding their nature as well as their prevalence in the evolution of such complex systems, and change impact analysis in variant-rich systems. We designed a model-based approach to extract feature related changes in heterogeneous artefacts. With such an approach, we can gather detailed information of feature evolution in all relevant artefacts. We created an approach for multi-product line modelling for impact computation. We leverage variability information to produce a collection of inter-related variability models, and show how to use it for targeted feature- change impact analysis on available capabilities of the product family. By applying our change extraction approaches on the Linux kernel, we were able to empirically characterise the evolution of the variability model of this system. We showed that the variability model of that system evolves mostly through modification of existing features, rather than through additions and removals. Similarly, we studied co-evolution of artefacts dur- ing feature evolution in the Linux kernel. Our study revealed that, in this system, most features evolve mostly through their implementation, and complex changes, involving heterogeneous artefacts are not the most frequent. Through this work, we provide detailed information on the evolution of a system, namely the Linux kernel, and the means used to obtain this information. We show that the gathered data allow us to reflect on the evolution of such a system, and we argue that gathering such information on any system is a source of valuable information regarding a system architecture. To this end, all tools developed in the context of this study were made available to the public. In this work, we provide key information on the evolution of the Linux kernel, as well as the means to obtain the same information from other variant-rich systems. The knowledge gained on common evolution scenarios is critical for tool developers focus- ing on the support of development of variant-rich systems. A better understanding of common evolution scenario also allows engineers to design systems that will be better equipped to elegantly evolve through such scenarios. While a number of challenges will still have to be addressed in this domain, this work constitutes a step toward a better understanding of variant-rich system evolution and therefore toward better variant-rich system designs. @en

Evolving a large scale, highly variable system is a challenging task. For such a system, evolution operations often require to update consistently both their implementation and its feature model. In this context, the evolution of the feature model closely follows the evolution of the system. The purpose of this work is to show that fine-grained feature changes can be used to guide the evolution of the highly variable system. In this paper, we present an approach to obtain fine-grained feature model changes with its supporting tool “FMDiff”. Our approach is tailored for Kconfig-based variability models and proposes a feature change classification detailing changes in features, their attributes and attribute values. We apply our approach to the Linux kernel feature model, extracting feature changes occurring in sixteen official releases. In contrast to previous studies, we found that feature modifications are responsible for most of the changes. Then, by taking advantage of the multi-platform aspect of the Linux kernel, we observe the effects of a feature change across the different architecture-specific feature models of the kernel. We found that between 10 and 50 % of feature changes impact all the architecture-specific feature models, offering a new perspective on studies of the evolution of the Linux feature model and development practices of its developers.

@en

The study of the evolution of highly configurable systems requires a thorough understanding of thee core ingredients of such systems: (1) the underlying variability model; (2) the assets that together implement the configurable features; and (3) the mapping from variable features to actual assets. Unfortunately, to date no systematic way to obtain such information at a sufficiently fine grained level exists. To remedy this problem we propose FEVER and its instantiation for the Linux kernel. FEVER extracts detailed information on changes in variability models (KConfig files), assets (preprocessor based C code), and mappings (Makefiles). We describe how FEVER works, and apply it to several releases of the Linux kernel. Our evaluation on 300 randomly selected commits, from two different releases, shows our results are accurate in 82.6% of the commits. Furthermore, we illustrate how the populated FEVER graph database thus obtained can be used in typical Linux engineering tasks.

@en

Linux is a highly configurable operating-system kernel which has been widely studied in the context of software product lines over the past years. Understanding the challenges and perils of evolving and maintaining feature models of the size of Linux is crucial to provide the right tools for development today and to direct future research. Unfortunately, previous studies show contradictory observations when analyzing the evolution of Linux feature models. We explain how peculiarities of the feature models of the Linux kernel lead to those differing observations, and show how the results can be re-aligned. Moreover, our findings also demonstrate that symbolic differencing on feature models used by researchers so far has limited value, depending on the use case. We show how the limitations can be addressed by means of semantic differencing, and ironically invalidate the results we sought to re-align.

@en

Variant-rich software systems offer a large degree of customization, allowing users to configure the target system according to their preferences and needs. Facing high degrees of variability, these systems often employ variability models to explicitly capture user-configurable features (e.g., systems options) and the constraints they impose. The explicit representation of features allows them to be referenced in different variation points across different artifacts, enabling the latter to vary according to specific feature selections. In such settings, the evolution of variability models interplays with the evolution of related artifacts, requiring the two to evolve together, or coevolve. Interestingly, little is known about how such coevolution occurs in real-world systems, as existing research has focused mostly on variability evolution as it happens in variability models only. Furthermore, existing techniques supporting variability evolution are usually validated with randomly-generated variability models or evolution scenarios that do not stem from practice. As the community lacks a deep understanding of how variability evolution occurs in real-world systems and how it relates to the evolution of different kinds of software artifacts, it is not surprising that industry reports existing tools and solutions ineffective, as they do not handle the complexity found in practice. Attempting to mitigate this overall lack of knowledge and to support tool builders with insights on how variability models coevolve with other artifact types, we study a large and complex real-world variant-rich software system: the Linux kernel. Specifically, we extract variability-coevolution patterns capturing changes in the variability model of the Linux kernel with subsequent changes in Makefiles and C source code. From the analysis of the patterns, we report on findings concerning evolution principles found in the kernel, and we reveal deficiencies in existing tools and theory when handling changes captured by our patterns.

@en

Despite a global recognition of the problem, and massive investment from researchers and practitioners, the evolution of complex software systems is still a major challenge for today's architects and developers. In the context of product lines, or highly configurable systems, variability in the implementation and design makes many of the pre-existing challenges even more difficult to tackle.Many approaches and tools have been designed, but developers still miss the tools and methods enabling safe evolution of complex, variable systems.In this paper, we present our research plans toward this goal: making the evolution of software product lines safer.We show, by use of two concrete examples of changes that occurred in Linux, that simple heuristics can be applied to facilitate change comprehension and avoid common mistakes, without relying on heavy tooling.Based on those observations, we present the steps we intend to take to build a framework to regroup and classify changes, run simple checks, and eventually increase the quality of code deliveries affecting the variability model, mapping and implementation of software product lines.

@en

Evolving large-scale, complex and highly variable systems is known to be a difficult task, where a single change can ripple through various parts of the system with potentially undesirable effects. In the case of product lines, and moreover multi-product lines, a change may affect only certain variants or certain combinations of features, making the evaluation of change effects more difficult.

In this paper, we present an approach for computing the impact of a feature change on the existing configurations of a multi-product line, using partial information regarding constraints between feature models. Our approach identifies the configurations that can no longer be derived in each individual feature model taking into account feature change impact propagation across feature models. We demonstrate our approach using an industrial problem and show that correct results can be obtained even with partial information. We also provide the tool we built for this purpose.@en