This thesis involves the development and testing of an educational tool named ”Playlist,” aimed at improving the learning experience of Computer Science & Engineering bachelor students at the TU Delft. The Playlist tool was developed as a feature for the existing tool called Skill Circuits, supporting students in creating personalized sequences of learning activities for their study sessions. This research contributes to the complexities of modern digital learning environments and the challenges students face in navigating vast amounts of educational content.

This report holds the full description of the integrated thesis. The first part including the chapters of ”Fostering a personal sequence of learning activities in Skill Circuits”.
The study uses a design-based research approach, which involves iterative development and research cycles and including students in the evaluation phases. By drawing on educational theories and addressing practical needs identified at TU Delft, the Playlist tool was designed to seamlessly integrate into the existing learning environment. The tool’s functionalities were prioritized based on user feedback provided by Computer Science students and gathered through surveys and focus group sessions.

Key findings indicate that playlists can improve student engagement, motivation, and overall learning experience by supporting students in finding learning activities aligned with their learning preferences and goals. The research highlights the importance of providing students with tools that improve navigating the learning environment and support them in self-directed learning. The Playlist feature not only aids students in organizing their learning activities but also promotes a holistic approach to education by incorporating supporting activities such as reflection and taking breaks.

The integrated thesis hopes to demonstrate the potential and importance of designing tools to personalize learning and support students in achieving their learning goals. It also explains why personalization tools in higher education should be implemented in an iterative manner.

The rising number of students in computer science presents a challenge for educators to analyse student work and identify problem areas at scale. Learning analytics systems often fall short in offering detailed, actionable insights that educators can use to enhance their teaching. To address this, we propose a system leveraging Large Language Models to analyse programming submissions and generate actionable analytics. The study focused on two primary research questions regarding the accuracy of LLMs in classifying programming submissions and identifying common issues, and educators' perceptions of the usefulness of the system. Through a system evaluation and focus group, we found that LLMs can analyse SQL assignment submissions with reasonable accuracy and can identify common issues. Educators found the insights potentially useful but noted areas for improvement such as the need for concrete statistics, accurate lists of submissions with specific issues, and more readable reports. Future research should address these implications for the system, evaluate the system in other courses with different programming languages, and involve stakeholders directly related to the course.

The online website Skill Circuits is a tool developed by teachers at the Delft University of Technology. Skill Circuits is an online learning tool that presents students with a node-link (i.e. a tree) structure where each node represents a skill, containing tasks that aim to teach the skill. The website aims to benefit teachers, by letting them think how all the 'skills' in their course relate and whether or not they have sufficient material to teach these skills. At the same time, the website aims to benefit students by giving them a learning path to follow, where each step is a small task, and each skill is visually connected to previous skills, showing how the learning material is connected.
This thesis aimed to help new teachers understand the concept of these node-link structures --- called skill circuits --- and also expand the website with features to help them evaluate their skill circuits, both before and during the course.
A documentation help page was developed, which explained the concept and intended usage of the website and was accompanied by a proof-of-concept skill circuit explaining the same topics. Tools to help evaluate a skill circuit were also developed, which included tools that could be used before the course was held, and tools that used analytics generated by students. One noteworthy tool took inspiration from the Constructive Alignment principle, where a teacher can label parts of their skill circuit with learning objectives to see if there is alignment with their course material.
All of these features were evaluated with a small focus group consisting of teachers, which showed enthusiasm toward the proposed features.

The rising number of students challenges the teacher’s time-consuming task to provide consistent and high-quality feedback for all students. To address the traditional education’s challenges, researchers refer to online educational tools to assist teachers. Although large writing tools (for instance, Grammarly and Microsoft Editor) assist students to write effectively, their primary objective is not to educate students. Therefore, we propose RevisionCoach – an automated writing feedback system that iteratively constructs educational, localized feedback to assist students to learn how to write. Clear and effective writing is important for students to succeed in academic endeavors and allows teachers to focus on feedback for the assignment’s primary task. RevisionCoach’s objective is to educate, and for that reason, the design considers learning by deliberate practice, differentiated learning, and self-regulated learning. In addition, RevisionCoach’s feedback has four layers: a sentence-level mistake highlight, an assessment category (coherence, cohesion, readability, and formality), a correction category (rewrite, reword, and rephrase), and a correction suggestion.

RevisionCoach’s categorized feedback allows for convenient evaluation that addresses RevisionCoach’s capability to predict the writing mistake’s importance. In the evaluation, we ask writing experts, students, Grammarly, and Microsoft Editor to find writing mistakes in text and to rate the mistake’s importance. Compared to the experts’ importance predictions, RevisionCoach predicts the mistake importances more accurately (1.89 mean square error (MSE)) than a random baseline (2.70 MSE), student baseline (2.50 MSE), Grammarly (2.52 MSE), and Microsoft Editor (2.02 MSE). Furthermore, the study illustrates the experts’ challenge to provide localized feedback for writing skills because the experts have merely 71.2% agreement about the top 2 most severe mistakes. At the same time, RevisionCoach achieves an average agreement of 72.9% with the experts’ mistake predictions.