Agent-based training systems can enhance people's social skills. The effective development of these systems needs a comprehensive architecture that outlines their components and relationships. Such an architecture can pinpoint improvement areas and future outlooks. This paper presents ARTES: a general architecture illustrating how components of agent-based social training systems work together. We studied existing systems and architectures for training and tutoring to design ARTES and identify its essential components and interaction characteristics. ARTES comprises two core components: the agent simulation of social situations, and educational elements to provide guided learning. We link ARTES's crucial components to four primary learning theories (behaviourism, cognitivism, social cognitive theory, and constructivism) to illustrate the role of agent simulation and tutoring elements in establishing desired learning outcomes. Furthermore, we map ARTES's components against eight architectures, 43 systems and three tools to indicate the components' relevance, completeness, generalisation, and deployment potential across contexts. In addition to ARTES, the paper also contributes by identifying future improvements and research directions, such as the agent's thinking, tutoring methods, knowledge transfer, and ethical implications. We believe ARTES can help bridge the gap between virtual human simulations and impactful educational learning, offering training system developers desirable features like understandability and adaptability.@en

Computational Thinking (CT) is considered a core 21st century digital skill. The aspect of assessment is crucial and knowing what, who, when, how, and where to assess is important for assessment design. In this study, we conducted an umbrella review to gain insights regarding CT assessment in higher education. In total, we analyzed 11 reviews, focusing on: (1) bibliographical and methodological characteristics of the reviews; (2) aspects relevant of assessment design, including a) assessed constructs, b) applied assessment methodologies, and c) assessment contexts. Our findings suggest an increased attention on this topic. However, hardly any reviews reasoned the selection of their review methodology, and most of the reviews did not thoroughly examine existing reviews. Regarding assessment design aspects, most reviews did not confine their scope to higher education; however, findings on interventions and educational settings show commonalities. We identified 120 unique assessed constructs and around 10 types of assessment methods. Though a combined use of distinct assessment methods is suggested in reviews, guidelines for appropriate assessment design are yet to be constructed. Based on the findings, we argue that it is necessary to explore different combinations of assessment design in various contexts to construct assessment guidelines.

@en

Computational Thinking (CT) is vital in today's digital era, especially in Engineering Education. While no official policy or teaching framework on CT education has been established in the Netherlands, a Western European country, there have been various initiatives for the integration of CT into the curriculum. Recognizing the crucial role of teachers in CT integration, we surveyed the perceptions and intentions of teachers in tertiary education in the Netherlands. Our survey encompassed two aspects: (1) teachers' perceptions of CT, and (2) their intentions to integrate CT into pedagogical activities. 38 teachers, mostly in Engineering Education, from across the Netherlands completed the questionnaire based on the UTAUT framework. Regarding CT perceptions, our investigation reveals that teachers possess an inadequate understanding of the relationship between CT and Computer Science, have limited training experiences in CT, and hold differing opinions on when and which constructs of CT should be integrated into different domains. Concerning teachers' intentions to integrate CT, the results exhibited a strong positive correlation between performance expectancy, attitude towards CT, and behavioral intention to implement CT in learning activities. To foster the integration of CT in tertiary education, our findings suggest the need for further development of higher education teacher training programs focused on CT and its relation to CS. Additionally, there is a call for further exploration of how to enhance teachers' performance expectancy and effort expectancy.@en

From an early age, girls may opt out of Computer Science (CS) for not fitting the CS stereotypes of being male, asocial and technology-oriented. These stereotypes might be strengthened by children's books on programming, but little is known about this. Therefore, this paper explores the gender, social interactions and interests of characters illustrated in ten popular extracurricular Scratch and Python children's books. We found more masculine than feminine characters in all but one book. Furthermore, nearly half of the characters are illustrated alone, and 15% are interacting with computers & robots. Over two-thirds of the characters fit at least one stereotypical trait. With this paper, we aim to create awareness of stereotypes in CS books among creators, publishers and buyers. Making and using more inclusive CS materials will help close the gender gap.

@en

Motivation. Variable naming practices are part of the software developer’s profession, influencing program comprehension and code quality. Yet, little is known about how variable naming practices are taught in beginner courses. Objective. This paper investigates naming beliefs, self-reported teaching practices, and observations regarding variable naming practices of teachers of introductory Python programming courses. Methods. We adopted an in-depth qualitative approach by interviewing ten teachers from secondary education and higher education and developed several themes in order to answer our research questions. Results. Among various opinions and practices, we found that teachers agree on using meaningful names, but have conflicting beliefs about what is meaningful. Moreover, the described teaching practices do not always match teacher’s views on meaningful names, and teachers rarely encourage students to use them. Instead, they express that naming practices should not be enforced and that students will develop them by example. Whereas some teachers report focusing solely on conventions, others deliberately dedicate time for students to engage with naming, create their own guidelines, provide continuous feedback, or include naming exercises on exams. Discussion. Naming practices do not seem to be deliberately taught, even though they influence program understanding and code quality. We also identified inconsistencies in teachers’ self-reported naming practices. As such, we encourage intentional conversations about naming practices in educational settings, specifically linking naming to code quality and readability. We see room for group and peer activities as a means to this end, as well as providing formative feedback dedicated to naming.

@en

Evaluation of educational innovation at the course level is dominated by frameworks that are linear, built on simple models, lack grounding in theory, and are generally not flexible enough to provide actionable recommendations for improvements. Moreover, the guidelines provided by existing literature are difficult to transfer to other contexts as most of these are aimed at specific innovations only. Therefore, there is a need for a new approach that is flexible enough to enable adaptation to different contexts, and that draws on the strengths of existing evaluation traditions. This paper forms a starting point for a larger research initiative for developing a comprehensive innovation evaluation framework. The context of the paper is a project based within the four universities of technology in The Netherlands (4TU) dedicated to strengthening collaboration in engineering education and research needed to take on global societal problems. The potential reach of our framework has implications for the wider field. This paper describes the results of an exploratory literature review to draw on the strengths of various evaluation theories to develop a new approach to evaluating educational innovation in courses. Value, Methods, and Use are the main components identified for this approach. Secondly, we developed a workshop to pilot this approach. Lastly, the workshop was presented to educational researchers, engineering educators, and educational advisors. We share lessons learned from the workshop and conclude with descriptions of future research to refine the framework to prepare its application in real innovation initiatives.@en

Describing and analysing learner behaviour using sequential data and analysis is becoming more and more popular in Learning Analytics. Nevertheless, we found a variety of definitions of learning sequences, as well as choices regarding data aggregation and the methods implemented for analysis. Furthermore, sequences are used to study different educational settings and serve as a base for various interventions. In this literature review, the authors aim to generate an overview of these aspects to describe the current state of using sequence analysis in educational support and learning analytics. The 74 included articles were selected based on the criteria that they conduct empirical research on an educational environment using sequences of learning actions as the main focus of their analysis. The results enable us to highlight different learning tasks where sequences are analysed, identify data mapping strategies for different types of sequence actions, differentiate techniques based on purpose and scope, and identify educational interventions based on the outcomes of sequence analysis. @en

Technology-enhanced learning systems, specifically multimodal learning technologies, use sensors to collect data from multiple modalities to provide personalized learning support beyond traditional learning settings. However, many studies surrounding such multimodal learning systems mostly focus on technical aspects concerning data collection and exploitation and therefore overlook theoretical and instructional design aspects such as feedback design in multimodal settings. This paper explores multimodal learning systems as a critical part of technology-enhanced learning used for capturing and analyzing the learning process to exploit the collected multimodal data to generate feedback in multimodal settings. By investigating various studies, we aim to reveal the roles of multimodality in technology-enhanced learning across various learning domains. Our scoping review outlines the conceptual landscape of multimodal learning systems, identifies potential gaps, and provides new perspectives on adaptive multimodal system design: intertwining learning data for meaningful insights into learning, designing effective feedback, and implementing them in diverse learning domains.

@en

Higher educational institutions have broadly adopted Collaborative Engineering Design (CED) activities to prepare students for complex problem-solving in multidisciplinary settings. These activities are non-linear and mediated by various social practices and tools. Therefore educators might struggle in facilitating the achievement of specific learning goals. Embodied cognition is an approach that explains non-linear behaviour through orgamism-environment interactions and might therefore provide educators with insights on how to prompt students towards desired actions in CED activities. According to embodied cognition, we learn through actions that emerge as a response to a problem (task) and environmental constraints. Educators can guide students’ behaviour by proposing tasks and adapting the environmental constraints of a learning situation, thus creating a field of promoted action. In this paper, we outline the progress of a design-based research in which insights from embodied cognition are implemented to promote desired student behaviour in CED activities. We report on the results of our problem-exploration phase. A systematic literature review and focus groups with students revealed that students are often hesitant to adopt new practices and tools that could potentially improve their collaborative design process. Next, we propose three theory-based design principles in which the task and environmental constraints are leveraged to foster the adoption of practices and tools and apply them to CED activities. Finally, we will share preliminary observations of the learning processes triggered by the designed activities and outline the directions for future research.@en

Women are underrepresented in Computer Science (CS). Closing the gender gap in CS benefits the economy, gender equality and society. However, girls have low CS interest while interest energizes learning and guides career trajectories. We explore the effects of the user interface of Hedy, a graduate language which teaches the syntax of Python, on the development of CS interest in girls.@en

The energy management systems industry in the built environment is currently an important topic. Buildings use about 40% of the total global energy worldwide. Therefore, the energy management system’s sector is one of the most influential sectors to realize changes and transformation of energy use. New data science technologies used in building energy management systems might not only bring many technical challenges, but also they raise significant educational challenges for professionals who work in the field of energy management systems. Learning and educational issues are mainly due to the transformation of professional practices and networks, emerging technologies, and a big shift in how people work, communicate, and share their knowledge across the professional and academic sectors. In this study, we have investigated three different companies active in the building services sector to identify the main motivation and barriers to knowledge adoption, transfer, and exchange between different professionals in the energy management sector and explore the technologies that have been used in this field using the boundary-crossing framework. The results of our study show the importance of understanding professional learning networks in the building services sector. Additionally, the role of learning culture, incentive structure, and technologies behind the educational system of each organization are explained. Boundary-crossing helps to analyze the barriers and challenges in the educational setting and how new educational technologies can be embedded. Based on our results, future studies with a bigger sample and deeper analysis of technologies are needed to have a better understanding of current educational problems.@en

Background: While research on Virtual Reality’s potential for education continues to advance, research on its support for Collaborative Learning is small in scope. With remote collaboration and distance learning becoming increasingly relevant for education (especially since the COVID-19 pandemic), an understanding of Virtual Reality’s potential for Collaborative Learning is of importance. To establish how this immersive technology can support and enhance collaboration between learners, this systematic literature review analyses scientific research on Virtual Reality for Collaborative Learning with the intention to identify 1) skills and competences trained, 2) domains and disciplines addressed, 3) systems used and 4) empirical knowledge established.Method: Two scientific databases—Scopus and Web of Science—were used for this review. Following the PRISMA method, a total of 139 articles were analyzed. Reliability of this selection process was assessed using five additional coders. A taxonomy was used to classify these articles. Another coder was used to assess the reliability of the primary coder before this taxonomy was applied to the selected articlesResults: Based on the literature reviewed, skills and competences developed are divided into five categories. Educational fields and domains seem interested in Virtual Reality for Collaborative Learning because of a need for innovation, communities and remote socialization and collaboration between learners. Systems primarily use monitor-based Virtual Reality and mouse-and-keyboard controls. A general optimism is visible regarding the use of Virtual Reality to support and enhance Collaborative LearningConclusion: Five distinct affordances of Virtual Reality for Collaborative Learning are identified: it 1) is an efficient tool to engage and motivate learners, 2) supports distance learning and remote collaboration, 3) provides multi- and interdisciplinary spaces for both learning and collaborating, 4) helps develop social skills and 5) suits Collaborative Learning-related paradigms and approaches. Overall, the reviewed literature suggests Virtual Reality to be an effective tool for the support and enhancement of Collaborative Learning, though further research is necessary to establish pedagogies.@en

Reading on digital devices has become more commonplace, while it often poses challenges to learners' attention. In this study, we hypothesized that allowing learners to reflect on their reading phases with an empathic social robot companion might enhance learners' attention in e-reading. To verify our assumption, we collected a novel dataset (SKEP) in an e-reading setting with social robot support. It contains 25 multimodal features from various sensors and logged data that are direct and indirect cues of attention. Based on the SKEP dataset, we comprehensively compared the difference between HRI-based (treatment) and GUI-based (control) feedback and obtained insights for intervention design. Based on the human annotation of the nearly 40 hours of video data streams from 60 subjects, we developed a machine learning model to capture attention-regulation behaviors in e-reading. We exploited a two-stage framework to recognize learners' observable self-regulatory behaviors and conducted attention analysis. The proposed system showed a promising performance with high prediction results of e-reading with HRI, such as 72.97% accuracy in recognizing attention regulation behaviors, 74.29% accuracy in predicting knowledge gain, 75.00% for perceived interaction experience, and 75.00% for perceived social presence. We believe our work can inspire the future design of HRI-based e-reading and its analysis.

@en

Motivation. Many people interested in learning a programming language choose online courses to develop their skills. The concept of variables is one of the most foundational ones to learn, but can be hard to grasp for novices. Variables are researched, but to our knowledge, few empirical observations on how the concept is taught in practice exist. Objective. We investigate how the concept of variables, and the respective naming practices, are taught in introductory Massive Open Online Courses (MOOCs) teaching programming languages. Methods. We gathered qualitative data related to variables and their naming from 17 MOOCs. Collected data include connections to other programming concepts, formal definitions, used analogies, and presented names. Results. We found that variables are often taught in close connection to data types, expressions, and program execution and are often explained using the 'variable as a box' analogy. The latter finding represents a stronger focus on 'storing values', than on naming, memory, and flexibility. Furthermore, MOOCs are inconsistent when teaching naming practices. Conclusions. We recommend teachers and researchers to pay deliberate attention to the definitions and analogies used to explain the concept of variables as well as to naming practices, and in particular to variable name meaning.

@en

Different individual features of the learner data often work as essential indicators of learning and intervention needs. This work exploits the personas in the design thinking process as the theoretical basis to analyze and cluster learners’ learning behavior patterns as groups. To adapt to the learning practice, we develop data-driven personas by clustering learners’ features based on factual learning outcomes (i.e., knowledge gain, perceived learning experience, perceived social presence) based on unsupervised learning, a more accessible and objective intervention design strategy for e-reading practices. Using the Chi-square test, we quantitatively evaluate different clusters driven by various unsupervised learning methods on the multimodal SKEP dataset. Furthermore, for a more practical real-life application, we achieved automatic persona prediction based on the attention regulation behaviors of learners. The subject-independent evaluation results indicate the best classification accuracy of 70% for the four-level classification task, differentiating three personas of learners with needs and another without feedback needs. It also shows that time-based sampling on both independent and cumulative learner behaviors works as robust predictors of learner personas, achieving a stable accuracy range of 65%-70% throughout the e-reading with the SVM classifier. Our work inspires the design of a real-time feedback loop for e-learning based on conversational agents.@en

This study is built upon a behavior-based framework for real-time attention evaluation of higher education learners in e-reading. Significant challenges in AI model developments for learning analytics have been 1) defining valid indicators and 2) connecting the analytics results to interventions, balancing the generalization and personalization needs. To address this, we utilized a public multimodal WEDAR dataset and trained a neural network model based on real-time features of learners, aiming at predicting learners’ moment-to-moment distractions. Real-time features for model training include 30 learners’ attention regulation behaviors annotated every second, reaction times to blur stimuli, and page numbers indicating various reading phases. Our preliminary model based on a neural network has achieved 66.26% accuracy in predicting self-reported distractions. Based on the model, we suggest a framework of a Behavior-based Feedback Loop for Attentive e-reading (BFLAe). It has text blur as feedback, a mechanism responsive to learners’ distractions that also works as data for next-round feedback. The general feedback implementation rules are established on a statistical analysis conducted on all learners. In addition, we propose a strategy for personalizing feedback using a quartile analysis of individual data, promoting learner-specific feedback. Our framework addresses the high demand for an automated e-learning assistant with non-intrusive data collection based on real-world settings and intuitive feedback provision. The feedback system aims to help learners with longer attention spans and less frequent distractions, leading to more engaging e-reading. @en

Background and Context. Increasing gender diversity in the field of Computer Science (CS) benefits the economy as well as gender equality. However, several obstacles - including underdeveloped CS interests, lack of programming experience, and a misfit with the stereotypes of computer scientists - prevent women from entering the field. Although these barriers develop from an early age, research focused on children is limited. Furthermore, limited work is done within European countries. Objectives. In this study, we research the interest children aged 7 to 14 have in a CS career. Additionally, we look into whether children with different characteristics have a different interest in CS. As such our research question: How does children's interest in a CS career differ based on their a) age, b) gender, c) computer interests, d) programming experience, and e) stereotypical beliefs? Method. We collected data from 200 children in a science museum located in the Netherlands. We gathered data on their gender, age, computer interests, programming experience, stereotypical beliefs and interest in becoming a programmer - as representative of a CS career. We used self-reported closed questions and reduced-length Child Implicit Association Tests. Findings. In general, the participating children are not interested in a CS career. We did find several characteristics related to a higher interest in a CS career: being a boy, having an interest in computers or video gaming, gaining programming experience at home or at an out-of-school activity, and the belief that programmers are social. Implications. To increase the participation of women in CS, we suggest motivating girls more to engage in computer and programming activities - especially out-of-school activities - while ensuring that these activities are gender inclusive.

@en

Increasing student numbers in higher education, particularly in engineering and computer science, make it difficult for motivated lecturers to continue engaging in active teaching methods such as Flipped Classrooms and Work-Based Learning. In these settings, digital Peer Assessment can be one approach to provide effective and scalable feedback. In Peer Assessment, students assess each other’s performance whilst gaining useful reflection and judgment skills at the same time. This umbrella review of 14 review papers on the use of (digital) Peer Assessment in education provides a comprehensive overview of design choices and their consequences open to educational practitioners wishing to implement digital Peer Assessment in their courses, the type of tooling available and the possible effects of these choices on the learning outcomes as well as potential pitfalls and challenges when implementing Peer Assessment. The paper will inform and assist educators in finding or developing a tool that fits their needs.@en

Contribution: This article presents a comprehensive overview of characteristics of educational designs of collaborative engineering design activities found in literature and how these characteristics mediate students' collaboration. Background: Engineers have to solve complex problems that require collaboration. In education, various collaborative engineering design activities have been implemented to prepare students for these professional practices. According to cultural historical activity theory (CHAT), educational activities can be described in terms of interrelated elements, i.e., subject, object, tools, rules, division of labor, and community, that influence learning outcomes. A key issue is how these elements mediate students' collaborative efforts and how they contribute to learning. Research Questions: 1) How is collaborative learning implemented in engineering design education? 2) How do the elements of CHAT and their interrelations mediate collaborative learning? and 3) What is the evidence that the implementation of collaborative learning contributed to the achievement of desired learning outcomes? Methodology: A systematic literature review following preferred reporting items for systematic review and meta-analyses protocols guidelines was conducted, including 111 articles published between 2011 and 2021. CHAT was used as analytical framework. Findings: Collaborative learning was implemented in engineering design activities to develop technical as well as nontechnical skills. For the CHAT elements, it was found that establishing a common object, rules for collaboration, and division of labor are essential for effective collaboration and can be enhanced through digital technologies (tools) and support from a community, for example, educators. Finally, results showed that there is evidence that described implementations contribute to learning. However, this evidence needs to be interpreted with care, due to methodological issues in some included articles.

@en