Development of a Generative AI-powered Teachable Agent for Middle School Mathematics Learning: A Design-based Research Study

This paper reports on a design-based research (DBR) study that aims to devise an artificial intelligence (AI)-powered teachable agent that supports secondary school students’ learning- by-teaching practices of mathematics learning content. A long-standing pedagogical practice of learning-by-teaching is powered by a recent advancement of generative AI technologies, yielding our teachable agent called ALTER-Math. This study chronicles one usability testing and three cycles of iterative design and implementation process of ALTER-Math. The three classroom studies involved a total of 320 middle school students and six teachers. The first study was exploratory, focusing on the qualitative feedback from the students and teachers through open- ended surveys, interviews, and classroom observations. The second study yielded a medium-high (M=3.26) quantitative survey result on students’ perceived engagement and usability on top of the qualitative findings. Lastly, the final study included pre- and post-knowledge tests in a quasi- experimental study design as well as student and teacher interviews. The final study revealed a bigger significant knowledge improvement in students who used ALTER-Math compared to the control group. The design implications learned from multiple iterations are discussed to inform the future design of AI-powered learning technologies.