Development Of A Generative AI-Powered Teachable Agent For Middle School Mathematics Learning: A Design-Based Research Study Abstract This paper reports on a design-based research (DBR) study that aims to devise an artificial intelligence (AI)-powered teachable agent that supports secondary school students’ learning- by-teaching practices of mathematics learning content. A long-standing pedagogical practice of learning-by-teaching is powered by a recent advancement of generative AI technologies, yielding our teachable agent called ALTER-Math. This study chronicles one usability testing and three cycles of iterative design and implementation process of ALTER-Math. The three classroom studies involved a total of 320 middle school students and six teachers. The first study was exploratory, focusing on the qualitative feedback from the students and teachers through open- ended surveys, interviews, and classroom observations. The second study yielded a medium-high (M=3.26) quantitative survey result on students’ perceived engagement and usability on top of the qualitative findings. Lastly, the final study included pre- and post-knowledge tests in a quasi- experimental study design as well as student and teacher interviews. The final study revealed a bigger significant knowledge improvement in students who used ALTER-Math compared to the control group. The design implications learned from multiple iterations are discussed to inform the future design of AI-powered learning technologies.Keywords learning-by-teaching teachable agent generative AI mathematics learning AI-assisted learning environment
Analyzing Student Attention And Acceptance Of Conversational AI For Math Learning: Insights From A Randomized Controlled Trial
Analyzing Student Attention And Acceptance of Conversational AI For Math Learning: Insights From A Randomized Controlled Trial Abstract The significance of nurturing a deep conceptual understanding in math learning cannot be overstated. Grounded in the pedagogical strategies of induction, concretization, and exemplification (ICE), we designed and developed a conversational AI using both ruleand generation-based techniques to facilitate math learning. Serving as a preliminary step, this study employed an experimental design involving 151 U.S.-based college students to reveal students’ attention patterns, technology acceptance model, and qualitative feedback when using the developed ConvAI. Our findings suggest that participants in the ConvAI group generally exhibit higher attention levels than those in the control group, aside from the initial stage where the control group was more attentive. Meanwhile, participants appreciated their experience with the ConvAI, particularly valuing the ICE support features. Finally, qualitative analysis of participants’ feedback was conducted to inform future refinement and to inspire educational researchers and practitioners. Keywords Technology design and development, Conversational AI, Large language models, Math learningGeneralization, Natural language processing, CollaborationMultiple Choice Question, Large Language Models, Humanin-the-loop.-
Automated Feedback For Student Math Responses Based On Multi-Modality And Fine-Tuning
Automated Feedback For Student Math Responses Based On Multi-Modality And Fine-Tuning Abstract Open-ended mathematical problems are a commonly used method for assessing students’ abilities by teachers. In previous automated assessments, natural language processing focusing on students’ textual answers has been the primary approach. However, mathematical questions often involve answers containing images, such as number lines, geometric shapes, and charts. Several existing computer-based learning systems allow students to upload their handwritten answers for grading. Yet, there are limited methods available for automated scoring of these image-based responses, with even fewer multi-modal approaches that can simultaneously handle both texts and images. In addition to scoring, another valuable scaffolding to procedurally and conceptually support students while lacking automation is comments. In this study, we developed a multi-task model to simultaneously output scores and comments using students’ multi-modal artifacts (texts and images) as inputs by extending BLIP, a multi-modal visual reasoning model. Benchmarked with three baselines, we fine-tuned and evaluated our approach on a dataset related to open-ended questions as well as students’ responses. We found that incorporating images with text inputs enhances feedback performance compared to using texts alone. Meanwhile, our model can effectively provide coherent and contextual feedback in mathematical settings. Keywords Generalization, Natural language processing, Collaborationanalytiopen-ended response, auto-scoring, automated comment, image response, multi-modality, fine-tuning
Retrieval-Augmented Generation To Improve Math Question-Answering: Trade-Offs Between Groundedness And Human Preference
Interactive question-answering (QA) with tutors has shown to be an effective way for middle school math students to learn. While not all students have access to a tutor, large language models make it possible to automate portions of the tutoring process–including interactive QA to support students’ conceptual discussion of mathematical concepts. Some have questioned how LLM responses can be better aligned with a school’s curriculum. In this paper, Levonian and colleagues explore how retrieval-augmented generation (RAG) can help improve response quality by incorporating textbook information and other educational resources, while also identifying trade-offs of using RAG.