So how can you tell that Knewton is helping students learn better?
It’s never easy to show the efficacy of a given set of instructional materials, since many other factors contribute to how students learn, from how the teacher runs the classroom to what they had for breakfast. Beyond that, the impact of Knewton can be difficult to isolate, since Knewton adaptive learning technology is only one component of any of the digital learning products we power.
As Knewton powers more products and serves more personalized recommendations, however, our data scientists can identify more examples that show how Knewton improves student outcomes.
In “The Improvement Index: Evaluating Academic Gains in College Students Using Adaptive Lessons,” Illya Bomash and Chris Kish of Knewton look at interactions involving approximately 288,000 students. They found that students performed better on average in college-level science courses with Knewton-powered adaptive assignments than in those without.
During the summer and fall semesters of 2013, these students studied college-level physics, biology, chemistry, and anatomy and physiology. Their textbooks came with a Knewton-powered online homework tool.
Knewton didn’t power the entire tool. Instructors assigned homework as usual for the students to complete online. On top of that, however, instructors had the option of enabling Knewton-powered adaptive follow-up assignments for students whose homework didn’t show mastery of the scientific concepts.
During these semesters, about 6,400 courses used the online homework tool. About a quarter of those courses offered Knewton-powered adaptive assignments. The rest didn’t.
Bomash and Kish compared these two groups. Their analysis found that students performed better in courses with Knewton-powered adaptive assignments:
The improvement increases with more use of adaptive assignments. We saw a peak average score difference of four percentage points.
They explored several other possible explanations for the difference — perhaps the better performing group were better prepared, or more motivated because they didn’t want extra homework — and ruled those out in their analysis. What’s more, results from courses in 2014 show consistent improvement in scores:
In courses that relied most on Knewton-powered assignments, students spent less than 25 minutes per week on average doing this additional coursework.