Developing a Mastery Learning Service for ASSISTments
AI & Data Analytics
Overview & The Problem
ASSISTments is a widely used, free online education tool primarily designed for K-12 math instruction. It serves as a formative assessment platform that provides immediate feedback to students as they work through problems and offers real-time learning data to teachers. It also supports high-impact tutoring programs by integrating traditional learning materials with powerful digital tools.
The core challenge addressed by this project was to enhance personalized learning and provide more granular, data-driven insights for teachers, schools, and researchers. By implementing a mastery learning service that accurately tracks and predicts student knowledge in real-time, we aimed to move beyond basic assessment to proactive, tailored educational support. The goal was to bridge the gap between students' current understanding and their potential for mastery, ultimately freeing up valuable teacher time to focus on relationship-building and differentiated instruction.
My Role & Contributions
My primary role in this project involved the development, training, and comparative analysis of machine learning models to predict student skill mastery. My specific contributions included:
- Model Development & Comparison: I was responsible for implementing and rigorously comparing two distinct machine learning models: a Logistic Regression model (often used in Performance Factor Analysis - PFA) and a Hidden Markov Model (Bayesian Knowledge Tracing - BKT). This involved setting up the experimental framework and evaluating their predictive capabilities for our specific use case.
- Data Analytics & Preprocessing: I performed extensive data cleaning and analysis on the ASSISTments dataset using SQL in PostgreDB to prepare the data for model training. This ensured the reliability and accuracy of the input features.
- API Backend Development: I contributed to the development of a RESTful API backend in Python, which would serve the trained models and facilitate real-time mastery predictions within the ASSISTments system.
Technologies Used
- Programming Language: Python
- Machine Learning Libraries: scikit-learn, numpy
- Data Manipulation: pandas
- Database: PostgreSQL (for SQL querying and data management)
- Web Framework (for API): custom Python
Solution & Approach: Bayesian Knowledge Tracing (BKT)
Our primary predictive solution centered around the **Bayesian Knowledge Tracing (BKT) model**, a probabilistic model rooted in cognitive science that mimics real-life learning processes. BKT operates on two unobserved (hidden) states for any given student-skill pairing: either the student has **not yet learned** the skill, or they **have learned** it.
The model uses five core probabilities, learned from student interaction data, to track a student$apos;s knowledge state over time:
- Initial Knowledge (P(L0)): The probability that a student already knows a skill before starting.
- Learn (P(T)): The probability that a student transitions from an unlearned state to a learned state after a successful attempt.
- Forget (P(F)): The probability that a student transitions from a learned state back to an unlearned state (representing forgetting).
- Guess (P(G)): The probability that a student in an unlearned state correctly answers a question by guessing.
- Slip (P(S)): The probability that a student in a learned state answers a question incorrectly due to a momentary mistake (slipping).
By continuously updating these probabilities based on a student$apos;s sequence of correct or incorrect answers within ASSISTments, the BKT model provides a dynamic, real-time prediction of whether a student has mastered a particular skill. This allows for a granular understanding of individual learning progress.
Impact & Results (Intended)
The development and analysis of this mastery learning service, particularly the BKT model, yielded significant potential impacts aligned with our goals for enhancing educational outcomes:
For Students:
- Improved Learning Outcomes: By providing real-time insights into individual student mastery levels, the model could enable adaptive learning pathways, ensuring students receive targeted practice on concepts they haven't yet mastered. This was designed to lead to increased scores on subsequent assessments and faster skill acquisition.
- Personalized Support: The system's ability to identify specific knowledge gaps would allow for timely interventions and tailored resources, effectively reducing learning barriers and enhancing the student's overall educational experience.
For Teachers:
- Reduced Administrative Workload: The insights from the model could be translated into actionable data dashboards, allowing teachers to quickly identify students at risk of falling behind or those ready for advanced material. This streamlines the identification of student needs for personalized intervention, potentially saving teachers significant time on manual assessment analysis (e.g., contributing to a **reduction of 10-15 hours per week** on data review and lesson planning).
- Enhanced Instructional Strategies: Teachers would gain clearer, data-driven perspectives, allowing them to focus more time on cultivating deeper student-teacher relationships and providing differentiated instruction.
For ASSISTments (Platform/Researchers):
- Advanced Student Modeling: Our work contributed to the platform's ability to offer more sophisticated student modeling capabilities, directly supporting its mission to accelerate math achievement. This enhanced functionality could lead to **increased user adoption** by demonstrating cutting-edge pedagogical support.
- Research Insights: The comparative analysis of BKT with Performance Factor Analysis provided valuable research insights into the most effective predictive approaches for the large-scale ASSISTments dataset, informing future development.
- Ethical Considerations: A core component of our approach included a clear disclaimer stating that these predicted values are not meant to replace the crucial student-teacher relationship or serve as a basis for surveillance. Instead, they are intended purely as a compass to guide educators towards students who may be in need of extra support, reinforcing our human-first philosophy.
Visuals & Demos
Visual summary of methodology and findings
Live demonstration of the BKT model
Comparing Bayesian Knowledge Tracing and Performance Factors Analysis Models
BKT Mastery Predictions over 10 Questions