In a randomized controlled trial (RCT) conducted in partnership with Fab AI and supported by the Sierra Leone Ministry of Education, we evaluated the impact of AI-assisted Guided Learning on the math progress of 1,763 junior secondary students. The results indicate that AI can serve as a powerful pedagogical partner, enhancing teachers’ reach rather than replacing them. The data revealed that 91.4% of student interactions focused on building conceptual understanding rather than simply seeking answers. AI posed scaffolding questions in 76% of its messages, providing direct solutions in only 2%, ensuring that cognitive effort remained with the students.
The trial's success hinged on the collaboration between AI and educators, where teachers designed lessons, set objectives, and facilitated discussions. Feedback from teachers indicated that the AI tool also supported their professional growth, allowing them to discover new ways to explain concepts. Many transitioned from being “lecturers” to “facilitators” in the classroom.
Quantitative results showed that students using Guided Learning achieved a gain of +0.258 standard deviations in math scores compared to the control group, translating to approximately 1.2 to 1.7 years of typical learning progress. Those teachers who incorporated Gemini into about half their lessons saw even greater gains, approximately 1.8 to 2.5 years of progress. Engagement was notably high, with 69% of students meeting usage targets, far exceeding the typical five percent engagement rate in voluntary educational technology.
Moreover, students reported increased enjoyment in math and actively engaged with learning, with their questions shifting from seeking direct answers to skill-building inquiries. By the final week, skill-building questions rose to 90%.
To further understand Guided Learning’s impact, we are conducting additional pre-registered RCTs globally. We will also release a playbook on our RCT approach to assist others in conducting scalable studies. While the results are promising, they highlight the challenge of the “achievement gap,” indicating a need for tools that yield the greatest benefits for those who need help the most.
Looking ahead, we plan to expand these trials to other countries and delve deeper into areas such as metacognition and relational intelligence for a more holistic view of learning. By combining the relational foundation of a teacher-led classroom with the personalized capabilities of AI, we aim to ensure that technology serves as a bridge to meaningful learning opportunities for all.
We also received support from Google.org and the Gates Foundation for this trial, with contributions from EducAid, Laterite, and Oxford MeasurEd.
Blogger's Review: This research highlights the immense potential of AI in education, particularly in enhancing student autonomy and supporting teachers. Well-designed AI tools can effectively foster understanding and skill development, making them worthy of broader application in diverse educational contexts.