Where the Math Gap Is Widest: Measurement Skills Among Middle School Students New data from the 2024 National Assessment of Educational Progress (NAEP) reveals a startling reality: the widest achievement gaps for marginalized middle schoolers aren’t in algebra or geometry, but in the foundational area of measurement. In a new report, “Where the Math Gap Is Widest,” authors L Burleigh and Kumar Garg explore how these disparities threaten to limit future career opportunities in an increasingly technical world. Read the analysis below or download the full report to learn more about how LEVI Math is creating viable pathways toward improving math proficiency.
ICYMI: Beyond The Test Scores: How LEVI Is Addressing Math’s Deeper Challenge
This blog post is based on a column that originally ran in The 74 on Feb. 5, 2026. In a landscape often dominated by sobering headlines about national test scores, a recent op-ed of math achievement on the National Assessment of Education Progress (NAEP) by L Burleigh, unearthed an underlying issue: the “math problem” in American classrooms isn’t actually about the numbers. It’s about a fundamental gap in how students are taught to think. According to Burleigh, U.S. students are currently facing a “comprehension crisis.” For decades, the education system has leaned heavily on procedural fluency, or teaching students to follow a set of memorized steps to reach an answer. While this might help a student pass a specific quiz, that knowledge isn’t correctly applied in the context of a problem that doesn’t fit the expected pattern. When the “recipe” fails, the student is left without the conceptual tools to navigate the kitchen. Engineering a Solution: The LEVI “Moonshot” In a landscape often dominated by sobering headlines about national test scores, a recent op-ed of math achievement on the National Assessment of Education Progress (NAEP) by L Burleigh, unearthed an underlying issue: the “math problem” in American classrooms isn’t actually about the numbers. It’s about a fundamental gap in how students are taught to think. According to Burleigh, U.S. students are currently facing a “comprehension crisis.” For decades, the education system has leaned heavily on procedural fluency, or teaching students to follow a set of memorized steps to reach an answer. While this might help a student pass a specific quiz, that knowledge isn’t correctly applied in the context of a problem that doesn’t fit the expected pattern. When the “recipe” fails, the student is left without the conceptual tools to navigate the kitchen. Learn more about the LEVI Math cohort are doubling math learning, from AI chatbots to digital avatars. From Drills to Dialogue: The Role of AI A standout example of this work is LEVI grantee Eedi. Traditional math software often functions as a high-tech version of a flashcard by marking an answer “correct” or “incorrect” and moving on. Eedi, however, uses advanced AI to diagnose the why behind a wrong answer. By identifying specific misconceptions, the platform acts less like a grader and more like a tutor. It asks guiding questions that force students to explain their reasoning, helping them build the mental scaffolding necessary to understand the “why” behind the “how.” This approach mimics high-dosage human tutoring at a scale that was previously impossible, directly addressing the comprehension gap identified in the original article. Building for the Long Term The takeaway is clear: if we want to move the needle on national math proficiency, we must stop treating math as a series of hurdles to jump over and start treating it as a language to be understood. By applying the rigors of engineering to the science of learning, LEVI and its teams are ensuring that students are building the critical thinking skills they will need for a lifetime.
Eedi Showing How AI Tutoring Can Deliver Personalized Learning Safely And Effectively
Education technology applications are continually being invented, and recent AI innovations have only increased that rate of innovation. While there are widely-accepted approaches to evaluate stable, late-stage products (e.g., Randomized Controlled Trials), there is much less clarity about how to conduct these evaluations at earlier stages of development. Given the potential risks in AI-powered solutions due to potential hallucinations and concerns about fairness, early evaluations are more important than ever.
AI Tutoring Can Safely And Effectively Support Students: An Exploratory RCT In UK Classrooms
AI Tutoring Can Safely And Effectively Support Students: An Exploratory RCT In UK Classrooms LearnLM Team, Google & Eedi One-to-one tutoring is widely considered the gold standard for personalized education, yet it remainsprohibitively expensive to scale. To evaluate whether generative AI might help expand access to thisresource, we conducted an exploratory randomized controlled trial (RCT) with 𝑵 = 165 students acrossfive UK secondary schools. We integrated LearnLM—a generative AI model fine-tuned for pedagogy—intochat-based tutoring sessions on the Eedi mathematics platform. In the RCT, expert tutors directlysupervised LearnLM, with the remit to revise each message it drafted until they would be satisfiedsending it themselves. LearnLM proved to be a reliable source of pedagogical instruction, with supervisingtutors approving 76.4% of its drafted messages making zero or minimal edits (i.e., changing only one ortwo characters). This translated into effective tutoring support: students guided by LearnLM performedat least as well as students chatting with human tutors on each learning outcome we measured. Infact, students who received support from LearnLM were 5.5 percentage points more likely to solvenovel problems on subsequent topics (with a success rate of 66.2%) than those who received tutoringfrom human tutors alone (rate of 60.7%). In interviews, tutors highlighted LearnLM’s strength atdrafting Socratic questions that encouraged deeper reflection from students, with multiple tutors evenreporting that they learned new pedagogical practices from the model. Overall, our results suggestthat pedagogically fine-tuned AI tutoring systems may play a promising role in delivering effective,individualized learning support at scale. Keywords learning, efficacy, safety, artificial intelligence, tutoring, randomized controlled trial
A Study To Evaluate The Effectiveness Of Eedi On Raising Attainment In Mathematics At KS3 (Year 7)
A Study To Evaluate The Effectiveness Of Eedi On Raising Attainment In Mathematics At KS3 (Year 7) Executive Summary This report evaluates the impact of Eedi, a digital mathematics platform, on raising maths attainment among Key Stage 3 students (Year 7). Eedi is an innovative EdTech platform offering over 60,000 diagnostic maths questions and engaging approximately 15,500 monthly users through its school and tutoring services. Enhanced by AI and video explanations, Eedi helps learners identify and address misconceptions while providing targeted additional support. Premium users also benefit from live, one-on-one tutoring, enabling students to request and receive assistance through the platform’s chat function. The impact evaluation included 20 schools, with randomisation conducted at the school level. The Eedi programme was implemented in intervention schools from Autumn 2023 to June 2024 to supplement classroom teaching, while control schools did not use the programme. Teachers assigned Eedi activities for completion either during lesson time or as additional homework tasks. The evaluation also incorporated a light-touch implementation and process review, drawing on online teacher surveys and Eedi usage data. Keywords large language models, intelligent tutoring systems, safety, system designGeneralization, Natural language processing, CollaborationMultiple Choice Question, Large Language Models, Humanin-the-loop.-
Designing Safe And Relevant Generative Chats For Math Learning In Intelligent Tutoring Systems
Designing Safe And Relevant Generative Chats For Math Learning In Intelligent Tutoring Systems Abstract Large language models (LLMs) are flexible, personalizable, and available, which makes their use within Intelligent Tutoring Systems (ITSs) appealing. However, their flexibility creates risks: inaccuracies, harmful content, and non-curricular material. Ethically deploying LLM-backed ITSs requires designing safeguards that ensure positive experiences for students. We describe the design of a conversational system integrated into an ITS that uses safety guardrails and retrieval-augmented generation to support middle-grade math learning. We evaluated this system using red-teaming, offline analyses, an inclassroom usability test, and a field deployment. We present empirical data from more than 8,000 student conversations designed to encourage a growth mindset, finding that the GPT-3.5 LLM rarely generates inappropriate messages and that retrieval-augmented generation improves response quality. The student interaction behaviors we observe provide implications for designers—to focus on student inputs as a content moderation problem—and implications for researchers—to focus on subtle forms of bad content and creating metrics and evaluation processes. Keywords large language models, intelligent tutoring systems, safety, system designGeneralization, Natural language processing, CollaborationMultiple Choice Question, Large Language Models, Humanin-the-loop.-
Improving Hybrid Human-AI Tutoring By Differentiating The Human Tutor’s Role Based On Student Need
Improving Hybrid Human-AI Tutoring By Differentiating The Human Tutor’s Role Based On Student Need Hybrid human-AI tutoring research highlights the promise of scaling personalized learning by combining adaptive AI tutor support with personalized instructional and motivational guidance from human tutors. However, emerging evidence suggests that human-AI tutoring is more beneficial for lower-performing students than higher-performing peers. This study evaluates a proactive-reactive personalization policy, where human tutors proactively initiate support for lower-performing students (below median), while higher-performing students receive reactive, on-demand support. Using a quasi-experimental difference-in-discontinuities design, 635 students (grades 5–8) were assigned to receive proactive or reactive tutoring during math practice using IXL (an AI tutor). Treatment assignment was determined using within-grade median state test scores. Results indicate that human-AI tutoring led to 43% (p = .01) growth on standardized tests compared to AI-only tutoring (i.e., 2.0× vs. 1.4× of expected growth). While the effects benefited both human-AI tutoring, the performance improvement was marginally greater than reactive tutoring (0.13 SD, p = .059). Beyond standardized tests, human-AI tutoring significantly increased students’ time-on-task (+1.38 hours, p < .001) and skill proficiency (+6.87 skills, p < .001) compared to AI-only tutoring. Finally, mediation analysis reveals a direct benefit of proactive tutoring on standardized test performance, despite lower time on task and skill proficiency compared to reactive tutoring—suggesting a “slow to go fast” dynamic where learning from direct interactions with tutors went beyond learning from IXL alone. Overall, the findings provide empirical evidence that access to human-AI tutoring leads to greater academic growth compared to AI-only tutoring, particularly for lower-performing students. More practically, these findings highlight a cost-effective strategy for scaling human-AI tutoring by allocating resources based on student needs, while improving learning for all students. Additional Keywords and Phrases Human-AI Tutoring, Learning Outcomes, Causal Inference
Improving The Validity Of Automatically Generated Feedback Via Reinforcement Learning
Improving The Validity Of Automatically Generated Feedback Via Reinforcement Learning Abstract Automatically generating feedback via large language models (LLMs) in intelligent tutoring systems and online learning platforms has the potential to improve the learning outcomes of many students. However, both feedback generation and evaluation are challenging: feedback content has to be valid especially in subjects like math, which requires models to understand the problem, the solution, and where the student’s error lies. Feedback also has to be pedagogically valid to reflect effective tutoring strategies, such as explaining possible misconceptions and encouraging the student, among other desirable features. In this work, we address both problems of automatically generating and evaluating feedback while considering both correctness and alignment. First, we propose a rubric for evaluating math feedback and show that GPT-4 is able to effectively use it to annotate human-written and LLM-generated feedback. Second, we propose a framework for feedback generation that optimizes both correctness and alignment using reinforcement learning (RL). Specifically, we use GPT-4’s annotations to create preferences over feedback pairs in an augmented dataset for training via direct preference optimization (DPO). We show that our methods significantly increase the correctness and alignment of generated feedback with Llama 2, an open-source LLM, qualitatively analyze our generation and evaluation systems using case studies, and outline several areas for future work Keywords Generalization, Natural language processing, CollaborationanalytiFeedback Generation, Human Preference Alignment, Math Education, Reinforcement Learning
Students’ Perceived Roles, Opportunities, And Challenges Of A Generative AI-Powered Teachable Agent: A Case Of Middle School Math Class
Students’ Perceived Roles, Opportunities, And Challenges Of A Generative AI-powered Teachable Agent: A Case Of Middle School Math Class Abstract Ongoing advancements in Generative AI (GenAI) have boosted the potential of applying long-standing “learning-by-teaching” practices in the form of a teachable agent (TA). Despite the recognized roles and opportunities of TAs, less is known about how GenAI could create synergy or introduce challenges in TAs and how students perceived the application of GenAI in TAs. This study explored middle school students’ perceived roles, benefits, and challenges of GenAI-powered TAs in an authentic mathematics classroom. Through classroom observation, focus-group interviews, and open-ended surveys of 108 sixth-grade students, we found that students expected the GenAI-powered TA to serve as a learning companion, facilitator, and collaborative problem-solver. Students also expressed the benefits and challenges of GenAI-powered TAs. This study provides implications for the design of educational AI and AI-assisted instruction. Keywords Generative AI in education; teachable agent; learning by teaching; AI-assisted instruction; Students’ perceptionsGeneralization, Natural language processing, CollaborationMultiple Choice Question, Large Language Models, Humanin-the-loop.-
Effective And Scalable Math Support: Experimental Evidence On The Impact Of An AI- Math Tutor In Ghana
Effective And Scalable Math Support: Experimental Evidence On The Impact Of An AI- Math Tutor In Ghana Abstract This study is a preliminary evaluation of the impact of receiving extra math instruction provided by Rori, an AI-powered math tutor accessible via WhatsApp, on the math performance of approximately 500 students in Ghana. Students assigned to both the control and treatment groups continued their normal classes with identical curricula and classroom hours. Students in the treatment group were given access to a phone for one hour a week – during their study hall period – and were allowed to use Rori to independently study math. All other aspects of the groups’ in-school experience were the same. We find that the math growth scores were substantially higher for the treatment group and statistically significant (p < 0.001). The effect size of 0.36 is considered large in the context of educational interventions: approximately equivalent to an extra year of learning. Importantly, Rori works on basic mobile devices connected to low-bandwidth data networks, and the marginal cost of providing Rori is approximately $5 per student, making it a potentially scalable intervention in the context of LMICs’ education systems. While the results should be interpreted judiciously, as they only report on year 1 of the intervention, they do suggest that chat-based tutoring solutions leveraging AI could offer a cost-effective and operationally efficient approach to enhancing learning outcomes for millions of students globally.Keywords LLMs, conversational agents, mobile-learning