Computational Thinking (CT) is considered a core 21st century digital skill. The aspect of assessment is crucial and knowing what, who, when, how, and where to assess is important for assessment design. In this study, we conducted an umbrella review to gain insights regarding CT assessment in higher education. In total, we analyzed 11 reviews, focusing on: (1) bibliographical and methodological characteristics of the reviews; (2) aspects relevant of assessment design, including a) assessed constructs, b) applied assessment methodologies, and c) assessment contexts. Our findings suggest an increased attention on this topic. However, hardly any reviews reasoned the selection of their review methodology, and most of the reviews did not thoroughly examine existing reviews. Regarding assessment design aspects, most reviews did not confine their scope to higher education; however, findings on interventions and educational settings show commonalities. We identified 120 unique assessed constructs and around 10 types of assessment methods. Though a combined use of distinct assessment methods is suggested in reviews, guidelines for appropriate assessment design are yet to be constructed. Based on the findings, we argue that it is necessary to explore different combinations of assessment design in various contexts to construct assessment guidelines.

@en

In recent years, researchers have developed several methods to automate discovering datasets and augmenting features for training Machine Learning (ML) models. Together with feature selection, these efforts have paved the way towards what is termed the feature discovery process. Data scientists and engineers use automated feature discovery over tabular datasets to add new features from different sources and enrich training data. By surveying data practitioners, we have observed that automated feature discovery approaches do not allow data scientists to use their domain knowledge during the feature discovery process. In addition, automated feature discovery methods can leak private features or introduce biased ones.

In this paper, we introduce the first user-driven human-in-the-loop feature discovery method called HILAutoFeat. We demonstrate the capabilities of HILAutoFeat, which effectively combines automated feature discovery with user-driven insights. Our demonstration is centred around two scenarios: (i) an automated feature discovery scenario -- HILAutoFeat acts as a steward in a large data lake where the user is unaware of the quality and relevance of the data, and (ii) a scenario where HILAutoFeat and the user work together -- the user drives the feature discovery process by adding his domain and business knowledge, while HILAutoFeat performs the intensive computations.@en

This report summarizes the outcomes of the second international workshop on Data Systems Education: Bridging Education Practice with Education Research (DataEd '23). The workshop was held in conjunction with the SIGMOD '23 conference in Seattle, USA on June 23, 2023. The aim of the workshop was to provide a dedicated venue for presenting and and discussing data management systems education experiences and research by bringing together the database and the computing education research communities to share findings, to crosspollinate perspectives and methods, and to shed light on opportunities for mutual progress in data systems education. The program featured two keynote talks, eight research paper presentations, and a discussion session. In this report, we present the workshop's main results, observations, and emerging research directions.@en

Computational Thinking (CT) is vital in today's digital era, especially in Engineering Education. While no official policy or teaching framework on CT education has been established in the Netherlands, a Western European country, there have been various initiatives for the integration of CT into the curriculum. Recognizing the crucial role of teachers in CT integration, we surveyed the perceptions and intentions of teachers in tertiary education in the Netherlands. Our survey encompassed two aspects: (1) teachers' perceptions of CT, and (2) their intentions to integrate CT into pedagogical activities. 38 teachers, mostly in Engineering Education, from across the Netherlands completed the questionnaire based on the UTAUT framework. Regarding CT perceptions, our investigation reveals that teachers possess an inadequate understanding of the relationship between CT and Computer Science, have limited training experiences in CT, and hold differing opinions on when and which constructs of CT should be integrated into different domains. Concerning teachers' intentions to integrate CT, the results exhibited a strong positive correlation between performance expectancy, attitude towards CT, and behavioral intention to implement CT in learning activities. To foster the integration of CT in tertiary education, our findings suggest the need for further development of higher education teacher training programs focused on CT and its relation to CS. Additionally, there is a call for further exploration of how to enhance teachers' performance expectancy and effort expectancy.@en

Multiple works in data management research focus on automating the processes of data augmentation and feature discovery to save users from having to perform these tasks manually. Yet, this automation often leads to a disconnect with the users, as it fails to consider the specific needs and preferences of the actual end-users of data management systems for machine learning. To explore this issue further, we conducted 19 semi-structured, think-aloud use-case studies based on a scenario in which data specialists were tasked with augmenting a base table with additional features to train a machine learning model. In this paper, we share key insights into the practices of feature discovery on tabular data performed by real-world data specialists derived from our user study. Our research uncovered differences between the user assumptions reported in the literature and the actual practices, as well as some areas where literature and real-world practices align.

@en

Motivation. Variable naming practices are part of the software developer’s profession, influencing program comprehension and code quality. Yet, little is known about how variable naming practices are taught in beginner courses. Objective. This paper investigates naming beliefs, self-reported teaching practices, and observations regarding variable naming practices of teachers of introductory Python programming courses. Methods. We adopted an in-depth qualitative approach by interviewing ten teachers from secondary education and higher education and developed several themes in order to answer our research questions. Results. Among various opinions and practices, we found that teachers agree on using meaningful names, but have conflicting beliefs about what is meaningful. Moreover, the described teaching practices do not always match teacher’s views on meaningful names, and teachers rarely encourage students to use them. Instead, they express that naming practices should not be enforced and that students will develop them by example. Whereas some teachers report focusing solely on conventions, others deliberately dedicate time for students to engage with naming, create their own guidelines, provide continuous feedback, or include naming exercises on exams. Discussion. Naming practices do not seem to be deliberately taught, even though they influence program understanding and code quality. We also identified inconsistencies in teachers’ self-reported naming practices. As such, we encourage intentional conversations about naming practices in educational settings, specifically linking naming to code quality and readability. We see room for group and peer activities as a means to this end, as well as providing formative feedback dedicated to naming.

@en

From an early age, girls may opt out of Computer Science (CS) for not fitting the CS stereotypes of being male, asocial and technology-oriented. These stereotypes might be strengthened by children's books on programming, but little is known about this. Therefore, this paper explores the gender, social interactions and interests of characters illustrated in ten popular extracurricular Scratch and Python children's books. We found more masculine than feminine characters in all but one book. Furthermore, nearly half of the characters are illustrated alone, and 15% are interacting with computers & robots. Over two-thirds of the characters fit at least one stereotypical trait. With this paper, we aim to create awareness of stereotypes in CS books among creators, publishers and buyers. Making and using more inclusive CS materials will help close the gender gap.

@en

In recent years, database education has been receiving more attention, with research in various directions such as the development of tools for education, the analysis of students’ homework, and the exploration of misconceptions. Misconceptions are mistakes in student reasoning that lead to errors during problem-solving. Recent work has documented misconceptions and errors in SQL. In this study we test the prevalence of several of these misconceptions through a multiple-choice questionnaire, to see if they hold on a larger, more diverse, student population. We found that all misconceptions are held to some extent, with prevalence scores ranging from one to fifty-two percent of the student population. Additionally, we have uncovered previously unidentified areas of struggle, allowing us to identify new misconceptions.@en

This report summarizes the outcomes of the first international workshop on Data Systems Education: Bridging Education Practice with Education Research (DataEd '22). The workshop was held in conjunction with the SIGMOD '22 conference in Philadelphia, USA on June 17, 2022. The aim of the workshop was to provide a dedicated venue for presenting and and discussing data management systems education experiences and research by bringing together the database and the computing education research communities to share findings, to crosspollinate perspectives and methods, and to shed light on opportunities for mutual progress in data systems education. The program featured two keynote talks, ten research paper presentations, a discussion session, and an industry panel discussion. In this report, we present the workshop's main results, observations, and emerging research directions.

@en

Background and Context. Increasing gender diversity in the field of Computer Science (CS) benefits the economy as well as gender equality. However, several obstacles - including underdeveloped CS interests, lack of programming experience, and a misfit with the stereotypes of computer scientists - prevent women from entering the field. Although these barriers develop from an early age, research focused on children is limited. Furthermore, limited work is done within European countries. Objectives. In this study, we research the interest children aged 7 to 14 have in a CS career. Additionally, we look into whether children with different characteristics have a different interest in CS. As such our research question: How does children's interest in a CS career differ based on their a) age, b) gender, c) computer interests, d) programming experience, and e) stereotypical beliefs? Method. We collected data from 200 children in a science museum located in the Netherlands. We gathered data on their gender, age, computer interests, programming experience, stereotypical beliefs and interest in becoming a programmer - as representative of a CS career. We used self-reported closed questions and reduced-length Child Implicit Association Tests. Findings. In general, the participating children are not interested in a CS career. We did find several characteristics related to a higher interest in a CS career: being a boy, having an interest in computers or video gaming, gaining programming experience at home or at an out-of-school activity, and the belief that programmers are social. Implications. To increase the participation of women in CS, we suggest motivating girls more to engage in computer and programming activities - especially out-of-school activities - while ensuring that these activities are gender inclusive.

@en

Motivation. Many people interested in learning a programming language choose online courses to develop their skills. The concept of variables is one of the most foundational ones to learn, but can be hard to grasp for novices. Variables are researched, but to our knowledge, few empirical observations on how the concept is taught in practice exist. Objective. We investigate how the concept of variables, and the respective naming practices, are taught in introductory Massive Open Online Courses (MOOCs) teaching programming languages. Methods. We gathered qualitative data related to variables and their naming from 17 MOOCs. Collected data include connections to other programming concepts, formal definitions, used analogies, and presented names. Results. We found that variables are often taught in close connection to data types, expressions, and program execution and are often explained using the 'variable as a box' analogy. The latter finding represents a stronger focus on 'storing values', than on naming, memory, and flexibility. Furthermore, MOOCs are inconsistent when teaching naming practices. Conclusions. We recommend teachers and researchers to pay deliberate attention to the definitions and analogies used to explain the concept of variables as well as to naming practices, and in particular to variable name meaning.

@en

Interest in data systems education is increasing, especially with the rise in demand for well-trained and re-trained data scientists. The database and the computing education research communities have complementary perspectives and experiences to share with each other. The DataEd workshop is organized as a dedicated venue for these communities to come together to share findings, cross-pollinate perspectives and methods, and shed light on opportunities for mutual progress in data systems education. In the DataEd workshop, we will present and discuss data management systems education experiences and research via keynotes and paper and poster presentations.

@en

Women are underrepresented in Computer Science (CS). Closing the gender gap in CS benefits the economy, gender equality and society. However, girls have low CS interest while interest energizes learning and guides career trajectories. We explore the effects of the user interface of Hedy, a graduate language which teaches the syntax of Python, on the development of CS interest in girls.@en

Querying a relational database is typically taught in practice by using an exercise database. Such databases may be simple toy examples or elaborate and complex schemas that mimic the real world. Which of these are preferable for students is yet unknown. Research has shown that while more complex exercise databases may hinder learning, they also benefit student engagement, as more complex databases are seen as more realistic. In our mixed-methods study, we explore what aspects of an exercise database contribute to student engagement in database education. To gain insight into what students would deem engaging, we asked 56 students to design, implement, and reflect on engaging databases for database education. The results imply that students are engaged by highly diverse yet easily understood database business domains, relatively simple database structures, and conceivable yet seemingly realistic amounts of data. The results challenge some previous study results while supporting approaches found in some textbooks, and provide guidelines and inspiration for educators designing exercise databases for querying and introducing relational database concepts.

@en

Several educational studies have argued for the contextualization of assignments, i.e., for providing a context or a story instead of an abstract or symbolic problem statement. Such contextualization may have beneficial effects such as higher student engagement and lower dropout rates. In the domain of database education, textbooks and educators typically provide an example database for context. These are then used to introduce key concepts related to database design, and to illustrate querying. However, it remains unstudied what kinds of database contexts are engaging for novices. In this paper, we study which aspects of database domain and complexity students find engaging through student reflections on a database creation assignment. We identify six factors regarding engaging domains, and five factors for engaging complexity. The main factor for domain-related engagement was Personal interest, the main factor for complexity engagement was Matching information requirements. Our findings can help database educators and book authors to design engaging exercise databases targeted for novices.

@en

Prior studies in the Computer Science education literature have illustrated that novices make many mistakes in composing SQL queries. Query formulation proves to be difficult for students. Only recently, some headway was made towards understanding why SQL leads to so many mistakes, by uncovering student misconceptions. In this article, we shed new light on SQL misconceptions by analyzing the hypotheses of SQL experts on the causes of student errors. By examining the experts' perceptions, we draw on their understanding of students' misconceptions and on their experiences with studying and teaching SQL. For our analysis, we chose the Policy Delphi, a questionnaire instrument specifically designed for gathering opinions and evidence. Through a two-round process, our nineteen participants proposed and voted on underlying causes for SQL errors which resulted in a set of hypotheses per error. Our main contribution to this article is this new set of possible misconceptions. With them, we can design more complete educational approaches to address misconceptions underlying SQL errors made by students, leading to more effective SQL education.

@en

Motivation Only 27% of computer and mathematical scientists in the United States and 18% of IT specialists in Europe are women. The under-representation of women in the field of Computer Science is, among other things, influenced by stereotypes of computer scientists. These stereotypes include being male, asocial and having an (obsessive) interest in computers. Even though stereotypical beliefs can develop at an early age, research on children’s stereotypes of computer scientists is sparse and inconclusive.
Objectives Stereotypes we hold can be implicit or unconscious beliefs, or explicit or conscious beliefs. In this study, we focus on children’s implicit and explicit stereotypes regarding computer scientists’ gender, social skills and interests. We also study whether explaining what a computer scientist does affects these stereotypes.
Method We study the implicit stereotypes through the reduced-length Child Implicit Association Test and the explicit stereotypes through self-reported absolute and relative Likert scale questions. We gathered data on 564 children between the age of 7 and 18 who were visiting a science museum. The participants in the experiment group (n=352) watch a video of either a man or woman explaining what a computer scientist does at the start of the study.
Results We found weak implicit stereotypical beliefs on computer scientists’ social skills and moderate implicit stereotypical beliefs on computer scientists’ interests. We also found explicit stereotypes on computer scientists’ gender, social skills and interests. Measuring the effects of the intervention, we found significant differences between the control and experiment group in their explicit stereotypes on computer scientists’ social skills.
Discussion The amount of scientific work on children’s stereotypes regarding computer scientists is still limited. Applying the reduced-length Child Implicit Association Test to measure children’s stereotypes on computer scientists has, to our knowledge, not been done before. Understanding children’s stereotypes and how to tackle them contributes to closing the gender gap in Computer Science.@en

When teaching novices programming, misconceptions can occur. Misconception are incorrect beliefs about certain programming concept. For example, some novices think that a variable can hold multiple values, in the case of two consecutive assignment statements, such as x = 5; x = 7. While explaining variables introductory materials often use the metaphor of a box for a variable, which might contribute to the 'multiple values' hypothesis. To investigate this, we design and run a controlled experiment with 496 novice programmers, both children and adults. Half of our participants receive an introductory programming lesson in which we explain a variable as a box, while the other half of participants receive the explanation of a variable as being a label. They are subsequently questioned about their understanding of variables. Our results show that, for the simple questions involving one assignment, the box group performs better. However, for questions involving the misconception - with two consecutive assignment statements - the label group outperforms the box group. This however primarily occurs when considering variables of type string, for integers subjects interpret the statements as numeric values to be added.

@en

When children learn to read, they almost invariably start with oral reading: reading the words and sentences out loud. Experiments have shown that when novices read text aloud, their comprehension is better then when reading in silence. This is attributed to the fact that reading aloud focuses the child’s attention to the text. We hypothesize that reading code aloud could support program comprehension in a similar way, encouraging novice programmers to pay attention to details. To this end we explore how novices read code, and we found that novice programmers vocalize code in different ways, sometimes changing vocalization within a code snippet. We are thus lead to believe that in order to teach novices to read code aloud, an agreed upon way of reading code is needed. As such, this paper proposes studying code phonology, ultimately leading to a shared understanding about how code should be read aloud, such that this can be practiced. In addition to being valuable as an educational and diagnostic tool for novices, we believe that pair programmers could also benefit from standardized communication about code, and that it could support improved tools for visually and physically disabled programmers.@en

Massive Open Online Courses (MOOCs) aim to educate the world. More often than not, however, MOOCs fall short of this goal — a majority of learners are already highly educated (with a Bachelor degree or more) and come from specific parts of the (developed) world. Learners from developing countries without a higher degree are underrepresented, though desired, in MOOCs. One reason for those learners to drop out of a course can be found in their financial realities and the subsequent limited amount of time they can dedicate to a course besides earning a living. If we could pay learners to take a MOOC, this hurdle would largely disappear. With MOOCS, this leads to the following fundamental challenge: How can learners be paid at scale? Ultimately, we envision a recommendation engine that recommends tasks from online market places such as Upwork or witmart to learners, that are relevant to the course content of the MOOC. In this manner, the learners learn and earn money. To investigate the feasibility of this vision, in this paper we explored to what extent (1) online market places contain tasks relevant to a specific MOOC, and (2) learners are able to solve real-world tasks correctly and with sufficient quality. Finally, based on our experimental design, we were also able to investigate the impact of real-world bonus tasks in a MOOC on the general learner population.@en