Background: Readability metrics provide us with an objective and efficient way to assess the quality of educational texts. We can use the readability measures for finding assessment items that are difficult to read for a given grade level. Hard-to-read math word problems can put some students at a disadvantage if they are behind in their literacy learning. Despite their math abilities, these students can perform poorly on difficult-to-read word problems because of their poor reading skills. Less readable math tests can create equity issues for students who are relatively new to the language of assessment. Less readable test items can also affect the assessment's construct validity by partially measuring reading comprehension. Objectives: This study shows how large language models help us improve the readability of math assessment items. Methods: We analysed 250 test items from grades 3 to 5 of EngageNY, an open-source curriculum. We used the GPT-3 AI system to simplify the text of these math word problems. We used text prompts and the few-shot learning method for the simplification task. Results and Conclusions: On average, GPT-3 AI produced output passages that showed improvements in readability metrics, but the outputs had a large amount of noise and were often unrelated to the input. We used thresholds over text similarity metrics and changes in readability measures to filter out the noise. We found meaningful simplifications that can be given to item authors as suggestions for improvement. Takeaways: GPT-3 AI is capable of simplifying hard-to-read math word problems. The model generates noisy simplifications using text prompts or few-shot learning methods. The noise can be filtered using text similarity and readability measures. The meaningful simplifications AI produces are sound but not ready to be used as a direct replacement for the original items. To improve test quality, simplifications can be suggested to item authors at the time of digital question authoring.

@en

Intelligent Tutoring Systems (ITS) can only respond adaptively to the digital learning activities of the students. If students are learning offline without any digital devices, they have little or no means to receive personalized learning materials with the help of intelligent systems. This paper proposes a Paper-Digital Integration System that can provide offline learners equitable access to ITS capabilities by looking at their work on paper and giving personalized printable feedback. We analyzed data from a paper algebra assessment of N = 17 students and found mistakes that may generalize and help us offer adaptive paper-based recommendations to students. Our analysis showed us some specific algebra mistakes that may help in providing intelligent feedback.

@en

Technology aided learning is becoming increasingly popular. In some of the countries, online learning has taken over for traditional classroom-based learning. With this, educational data is being generated in vast amounts. Knowing the potential of this data, many education stakeholders have turned to evidence-based decision making to improve the learning outcomes of the students. EdOptimize platform provides extensive actionable insights for a range of stakeholders through a suite of 3 data dashboards, each one intended for a certain type of stakeholder. We have designed a conceptual model and data architecture that can generalize across many different edtech implementation scenarios. Our source code is available at https://github.com/PlaypowerLabs/EdOptimize @en

Process Analysis is an emerging approach to discover meaningful knowledge from temporal educational data. The study presented in this paper shows how we used Process Analysis methods on the National Assessment of Educational Progress (NAEP) test data for modeling and predicting student test-taking behavior. Our process-oriented data exploration gave us insightful findings of how students were interacting with the digital assessment system over time. To discover what processes students were following during the NAEP Digital Assessment, we first developed an innovative set of research questions. Then, we used Process Analysis methods to answer these questions and created a set of features that described student behavior over time. These features were used to create an ensemble model that aimed to accurately predict the digital test-taking efficiency of the students taking NAEP. Our model emerged as one of the most successful models in the 2019 NAEP Data Mining Competition, scoring second place out of 89 teams.

@en

This paper examines the use of “pacing plots” to represent variations in student learning sequences within a digital curriculum. Pacing plots are an intuitive and flexible data visualizations that have a potential for revealing the diversity of blended classroom instructional models. By using curriculum pacing plots, we identified several common implementation patterns in real-world classrooms. After analyzing two years’ worth of data from over 150,000 students in a digital math curriculum, we found that a PCA and K-Means clustering approach was able to discover pedagogically relevant instructional practices.

@en