# Understanding Content for Efficient and Effective Learning

A thorough and nuanced understanding of educational content is central to the Knewton approach to adaptive learning. Every time we generate a recommendation for a student in a Knewton-enhanced course, we rely not only on data about that student’s activity, but also on a comprehensive understanding of the conceptual relationships that exist among the various pieces of content within the course. The Knewton Adaptive Instruction Team works closely with subject-matter experts at our partner companies to analyze content and represent these conceptual relationships. This helps us generate recommendations that allow students to learn more efficiently and effectively.

Here’s an example of how this works. Suppose that a student at the beginning of her first Knewton-enhanced course is struggling with a word problem which involves calculating the area of a triangle. Assume we know nothing about the student aside from this fact. (This is an uncommon scenario — students who have made any progress in a Knewton-enhanced course or who have taken previous Knewton-enhanced courses will have already generated proficiency data that can help inform recommendations.)  Knewton must determine why the student is getting this exercise wrong, so that we can recommend content that helps her learn the skills and concepts required to solve this problem.

Specifically, we must ask the following question: what is preventing the student from solving the triangle problem? There are several possibilities. It may be the case that she doesn’t know how to calculate the area of a triangle. Perhaps she struggles to read and interpret word problems. Maybe the base and the height of the triangle are given as decimals and she doesn’t know how to multiply decimals. There is even the possibility that she doesn’t know how to multiply integers!

It might also be the case that she can find the area of a hundred triangles with her eyes closed while she taps her head, rubs her belly and hops on one foot, and that she’s simply distracted by a computer game that she’s toggling to and from as her teacher wanders in and out of eyeshot of her computer screen. This last possibility is an important one. However, for the purposes of this example, let’s assume that the student is engaged and that her difficulty stems from the fact that she doesn’t understand one or more of the skills or concepts I described above. As you’ll recall if you’ve read one of our posts on knowledge graphs or checked out our white paper, we refer to these concepts as prerequisites.

Let’s list the prerequisites for the triangle word problem again:

1. Multiply integers
2. Multiply decimals
3. Calculate the area of a triangle
4. Read and interpret word problems

Using circles to represent the concepts and arrows to represent the prerequisite relationships, we can draw a diagram:

After we’ve identified the prerequisites, we must then assess the student’s proficiency in these areas so that we can recommend content that helps her learn the necessary concepts to solve the triangle word problem. For example, if she does poorly in assessments on calculating the area of a triangle, we can recommend additional content that helps her master this concept. But where do we start? Should we start by giving her an assessment on prerequisite 3 (Calculate the area of a triangle), or should we start with something more basic, like prerequisite 1 (Multiply integers)?

As you ponder this question, you might notice that the prerequisites in this example are not necessarily independent of one another. For example, it is unlikely that the student can multiply decimals if she cannot multiply integers. In fact, the content in this course that is associated with multiplying decimals expects and assumes that the student is able to multiply integers. In other words, 1 is a prerequisite for 2! Furthermore, prerequisite 1 (Multiply integers) is only important to the triangle problem as it relates to prerequisite 2 (Multiply decimals). In this case, we say that prerequisite 2 subsumes prerequisite 1 because it transmits the knowledge from 1 that is required to solve the triangle problem.

We can adjust our diagram to reflect this as follows:

How does this observation about the relationship between prerequisites 1 and 2 help us determine what the student does and does not know? Let’s imagine that we give the student an assessment on prerequisite 1 (Multiply integers) and she aces it. All we can say is that she’s proficient in prerequisite 1. However, what if we give her an assessment on prerequisite 2 (Multiply decimals) and she aces that? Since we know that the assessments for prerequisite 2 expect and assume that the student is proficient in prerequisite 1, then based on her performance in prerequisite 2, we can be fairly confident that she is proficient in both 1 and 2. Conversely, if she fails prerequisite 1 (Multiply integers), it’s probably safe to say that she is not proficient in prerequisite 2 (Multiply decimals) either. In other words, we can estimate her proficiency in certain concepts without having to directly assess her on them.

There is also the case that the student knows how to multiply integers but does not know how to multiply decimals. We can only determine this by assessing her on both concepts, and therefore, the subsumption relationship does not help us in this scenario. We can, however, use information about how similar students performed in the past to help us identify this scenario. (In a future blog post, we’ll expand on network effects, or how we utilize data about other students’ activity to inform the recommendations we generate for each individual student.)

The example above involves just a few concepts, but for a typical Knewton-enhanced course, we map out the relationships between hundreds of concepts. Knowing what concepts subsume other concepts allows us to eliminate concepts that we think students are already proficient in and more quickly hone in on what they should study to meet their goals. It’s like when the blocks disappear in Tetris!

To summarize: rather than assessing a student on every single prerequisite concept, we can use our understanding of the content  — specifically, the relationships that exist between the concepts — to make intelligent inferences about what the student does and does not know. As a result, we can generate recommendations that lead to a more efficient use of the student’s time and energy.

In this post, I’ve described one way that we use our understanding of content to make learning more efficient and effective for our students. In a future blog post, I’ll talk about how we use goals that are defined by students and instructors to help us generate better recommendations.

For more from the Knewton Adaptive Instruction team, check out Jesse Sternberg’s post on the cross-disciplinary approach of the Knewton knowledge graph and Matt Busick’s post on the power of a knowledge graph.

# The Power of a Knowledge Graph

I’d like to paint a scenario that reflects a common occurrence in classrooms nationwide. A student in an introductory algebra class — we’ll call him Stu — receives a score of 63%, a D, on his quiz from chapter 13 on adding and subtracting polynomials. The teacher — let’s call her Mrs. T — has several decades of experience under her belt and knows that if Stu simply moves on to the next topic without developing a proficient understanding of the skill of adding and subtracting polynomials, he will probably do poorly in much of the rest of the course. Having designed the quiz herself, she’s aware that it’s assessing several key objectives. Among these are:

1. Be familiar with the notation for adding and subtracting polynomial functions.
2. Add and subtract polynomials and polynomial functions.
3. Solve perimeter word problems involving the addition or subtraction of polynomials.
4. Solve other application problems that include histograms and charts by adding or subtracting polynomials.

Mrs. T is also aware that the problems on this quiz call upon and use previous skills in her course, skills she refers to as the section’s prerequisites. For example, a section from Chapter 9 introduced word problems involving perimeter, a Chapter 11 section covered how to interpret histograms and charts, a Chapter 12 topic provided a basic understanding of functions, and an earlier section in Chapter 13 covered combining like terms.

Mrs. T wants to help Stu review these prerequisite skills, so she examines her gradebook to see what scores he received in each topic. He seemed to do fairly well (over 80%) on the quiz from Chapter 9 on perimeter word problems and the quiz from Chapter 12 on functions, but he struggled (lower than 60%) with the section from Chapter 11 on interpreting histograms and charts and the earlier section in Chapter 13 on combining like terms.

Mrs. T creates a packet of review materials and puts together a remedial quiz containing questions from those sections in Chapters 11 and 13. Her hope is that after Stu has gained proficiency in the prerequisite skills, he will better understand the Chapter 13 material and be ready to re-tackle the quiz on adding and subtracting polynomials.

This sort of personalized remediation is indeed carried out by the best teachers in our schools, but it can be especially difficult if the student continues to fail in upcoming topics, and it becomes a Herculean, if not an impossible, task if the teacher has to carry it out for all the students in all her classes.

In a Knewton-powered adaptive course, the intricate scenario just outlined happens at the click of a button — thanks in part to the Knewton knowledge graph. The knowledge graph is a cross-disciplinary graph of academic concepts; within the graph, concepts have prerequisite relationships that help define a student’s path through the course.

When a student fails a topic in a Knewton-powered course, he or she is instantly remediated with prerequisite skills, prioritized on the strength of their relationships to the topic at hand and on the student’s demonstrated strengths and weaknesses. This frees Mrs. T from the administrative work of locating all prerequisite skills and correlating them with each student’s past performance. She now has more time to orchestrate classroom activities, introduce creative group work, or sit down with each student to address their misconceptions and encourage them through their frustrations.

Knewton’s knowledge graphs, carefully constructed by subject matter experts, incorporate the connections identified by experienced teachers into the course itself. Learning is by nature an extremely interrelated activity, and with a knowledge graph an adaptive platform can take full advantage of those connections when scaffolding students and guiding them toward mastery.

As more and more students progress through a Knewton course, the strength of these connections are refined over time. We may find that some prerequisite skills are rarely helpful, or only helpful to certain types of students with identified weaknesses, while others are extremely effective as skills to review before students return to a failed topic. The goal of such data-driven analytics is to mimic in real time, on a large scale, the sort of intuition a great teacher develops over his or her career. (For more on how Knewton uses student performance data to improve its recommendations over time, download the Knewton adaptive learning white paper).

Creating unique study plans for each student in a class would be incredibly time-consuming for teachers. By mapping each course to a continuously refined knowledge graph, Knewton does this automatically, ensuring that no student is left behind.

# The Knewton Knowledge Graph: A Cross-Disciplinary Approach

It’s an unavoidable problem in traditional classrooms: teachers, in the challenging position of teaching a class of 30 students or more with a wide variety of abilities and needs, are struggling to understand exactly what those needs and abilities are and to provide the instruction that will be most helpful for each student.

Knewton is working hard to solve this problem. In particular, we on the Knewton Adaptive Instruction Team have worked to create a system that assesses the needs of each individual student and serves him or her the learning experience he or she needs at exactly the right time.

To do this, we use the Knewton knowledge graph, a cross-disciplinary graph of academic concepts. Within the knowledge graph, concepts have prerequisite relationships that help define a student’s path through the course. Special relationships that define content as either “instructional” or “assessment” determine what kind of content to deliver to students at any given point. Knewton recommendations steer students on personalized and even cross-disciplinary paths on the Knowledge Graph towards ultimate learning objectives based on both what they know and how they learn. (For more about how cross-disciplinary learning paths are enhanced by continuous adaptivity and network effects, download our white paper on the science behind recommendation.)

Our team is always working to enhance the graph’s capacity to make fine-tuned recommendations for all the courses it powers. Often, this simply involves helping partners build graphs that better represent their content, but sometimes it can involve making changes to the nature of the graphing process itself on our end in order to ensure that the process is attuned to and capable of capturing the idiosyncrasies of a range of content domains.

Recently, we have used the latter process to help solve another challenge that inhibits learning in today’s classrooms. Since each subject in a traditional school requires a different teacher with the correct area of expertise, the various subjects are presented to students as being far more distinct and separate from each other than they actually are. Extensive studies show that students benefit in many ways from a cross-disciplinary approach, but practical concerns often get in the way. A history teacher might notice that her students’ essays suffer more from a lack of basic writing skills than from a lack of understanding of the historical facts, but she can’t suspend her own curriculum to teach those skills (even if she’s qualified to do so) and she can’t ask the English teacher to revisit them in his class, because he has to get his class through Hamlet by the end of the week. (It is true that some schools are advancing cross-disciplinary instruction and encouraging teachers of different disciplines to plan their curricula together; however, these schools are the exception, not the rule.)

Knewton’s goal is to link multiple subjects into one huge knowledge graph, rather than creating several separate ones in parallel. Generally, the process of graphing involves asking ourselves questions about the content at hand. If we’re looking at a section of a history book, for example, we would ask ourselves what other historical facts and concepts a student must understand in order to contextualize the new content. To create an interdisciplinary graph, we would ask questions like, “What reading level is necessary to parse out all the important details in this section?” and “What understanding of fractions and percentages is necessary to read the pie chart on page 145?”

The goal of this approach is to ensure that students possess the skills and knowledge they need to tackle the learning experience we recommend for them. We also hope that this will prove to be a helpful tool for educators to develop holistic curricula and collaborate effectively with their peers teaching other subjects. We hope that those who have not begun to approach learning in this way will now find it easier to begin to do so and that those who have already been struggling with these issues will see us a valuable ally.