Knerd Story – Josh Guillemette, Valencia College

Josh Guillemette
Statistical Methods and Math
Valencia College

How long have you been using Knewton Alta?

Five years.

How are you teaching?

I teach in all modalities. Fully online and then what we call blended learning, where we meet face-to-face once a week, and the rest of the instruction is online. It’s almost like flipped. Most of the instruction is online, and then when students come to me, I just answer their questions and help them with their homework and send them off again unstuck.

And then I actually do have one class that’s fully face-to-face. We meet twice a week, all six of the students in the class because of social distancing.

Why did you start using Knewton Alta?

My Wiley rep stopped by my office and told me about the product. She explained it was adaptive and took students’ answers and figures out where they’re at and guides them along the path.

The price ($44) was appealing too. At the time we were using Pearson’s My Math Lab (MML), and a lot of people still use it. It’s around $104 at the bookstore. So, it’s a difference of $60. I thought if Knewton Alta is even half as good, it’s worth checking out. So, I got permission to do a pilot and I got four other faculty members to do it with me.

Two of us had half of our classes use Pearson, and half of them used Knewton Alta. And then I had two more Pearson, but they gave the same final, and then I had another person doing all Knewton Alta. So, there was quite a smattering of implementations, and we were able to collect and analyze some data.

For first-year statistics, we use the CAOS (Comprehensive Assessment of Statistics), which is a peer-reviewed statistical reasoning instrument for the final. I did an analysis and, obviously, the teaching made the most difference. But they were all really good instructors, so there was only a little difference. There wasn’t really much difference, statistically, between what the Knewton Alta folks did and the Pearson folks did. And Knewton Alta was $60 less. I think what I would say now is that because it was a pilot, we didn’t really understand fully how to implement Knewton Alta, and if we did it again the results would probably be different.

Why do I keep using it?

The cost was a lot less, and the students seem to be a lot happier. And, again, just using it has been a really positive experience.

I get unsolicited happy notes from students who are using the Knewton Alta. They’ll say things like, “Thank you so much. The adaptive thing takes a little bit of getting used to, but I really felt like it helped my learning and I really learned stuff. And the system is really good.”

I never get that from MML. If I do get any comments about MML it’s usually, “It’s broken, help me fix it!” And so that’s been frustrating. Over the last couple of years (I don’t know if it’s the pandemic, or scaling, or what has happened) it just seems like their MML infrastructure hasn’t really kept up.

They seem to have a lot more technical difficulties, and I don’t have the same issues with Knewton Alta.

Student Feedback

We use a lot of MML in the math department, which means some students come in to my class with a MML background or mindset. They think, “I do the problems. I get them, it checks a box. I’m done. It’s 20 problems. That’s it.”

Over the years I’ve been using Knewton Alta, I’ve gotten a lot better at explaining how it works to the students. I explain…that Knewton Alta is a little bit different. It adapts to the information that you give it. And now that I explain things, I don’t get as many questions or students complaining. If I do have a student who seems to be a little frustrated, it tends to be because they don’t know how many questions there are when they go into the system. And I always tell them there’s a recommended range up in the corner, and most of the time it’s 8 to 12.

And I tell them if you go past 24 questions, you need to get a tutor, reach out to me, do something— because you don’t want to just spin your wheels. And most of them don’t. I get a lot of positive feedback, and what they say mostly is, “I really feel like I learned the material.”

One of the other things that I like, and I think students like it too, is the number of questions in Knewton Alta. In MML, you can put a review in there, but there are only so many questions, and then the review is over. But the review center in Knewton Alta is infinite, it keeps pulling questions from the test pool over and over again, forever. You can practice for as long as you want.

Students will ask when they should stop. I tell them, “When you’re getting 80% of the questions correct, then take the exam, take the certification. But if you’re not getting 80% correct, then you’re not ready and you need to keep going.” And I think students really like that aspect as well. And maybe that fits into the “I really have to learn it” mindset, as opposed to just figure out what the question is asking, check the right box, and move on.

How do you implement Knewton Alta?

I try to have students do something every day. So, there’s a due date and typically they’ll do three Knewton Alta assignments a week, sometimes two if they’re a little harder or longer. And then the week when we’re doing means, standard deviations, and Z scores, because it’s really just super basic calculations, they do four.

There is some level of instruction that I give them before they get started, whether it be a video or a Canvas page (which includes some classroom notes). I have 98 videos I’ve made for the stats class, and I can put a video in front of each of the lessons.

Students watch those things and then they’ll do the lesson. And then if they miss three or four questions in a row, it forces you to do the instruction again in Knewton Alta. If a student is struggling, I’ll remind them about the review instruction button. I tell them it might give them a slightly different point of view from what I’ve done in class, which might be helpful. And I know that Knewton Alta has made a change so now you can force the instruction first, before they get started. I don’t know how effective that is for most of my students. I think the original thought process behind how they set up Knewton Alta was the just-in-time instruction. Students typically dive in, get stuck and then try to figure it out.

So, I structured the class like that. Like, “Here’s some instruction. Here’s the homework. Go.”

I’ll get some students who will watch every minute of video twice, take lots of notes and then do the homework. But that’s the exception. The bell curve, the 68% in the middle, they’re just jumping in and when they get stuck, then they go back and read stuff until they feel like they’re not stuck. And then they’ll get stuck again. And then they’ll watch something. They go to, they get stuck, and it just I think that works for a lot of students.

In my math liberal arts class, I have fewer videos. And so I’m wondering about turning on the instruction beforehand for places where I haven’t made enough videos yet. So, that’s something that I’ve been thinking about so I’m glad that feature is there, but the original way that they set it up, like four years ago, I think that works for most students. And then the fact that it forces them to review instruction when they get four questions in a row wrong. Students need somebody to say, “Hey, this is how you’re supposed to do it!” And I think the students appreciate that because they’re not spinning their wheels forever.

I think that at the end of the day, you just have to meet students where they are. A lot of students just want to jump in. And so whether that’s good for them or not, I think you have to let them do it and then have a safety net underneath them.

Students come in at different levels. A lot of the students have never had stats, but then I have students who have had AP stats and they may remember some of that. So, they blow through the first module pretty easily. And there are students who are taking business calculus and their quantitative reasoning is at a level where it’s pretty quick for them to catch onto stuff. So, they don’t really struggle. And so for them, certainly not having to watch everything and read everything is helpful.

Is the adaptive nature of the program important to you?

Yes. Before the pandemic, they had a real-life Knerd camp up at New York. I got to go up there and talk to the data scientists about how the adaptive algorithm works, how they tried to make sure it never becomes a black box, and that there’s a person who has their hand on the wheel a little bit.

And it was a really good talk for me to understand how the algorithm works for the students. And it’s a big appeal for me. I really want the students to learn the material and develop some proficiency as opposed to “Do 20 problems, hit submit, and get 10 out of 10.” I really appreciate that aspect of Knewton Alta.

We’re allowed to adopt two textbooks for any one course on our campus. What initially drove the adoption of Knewton Alta as the second resource for statistics was the cost. And I think, being an advocate for it, that was a big driver. I think secondarily, the adaptive learning and the fact that it’s good software.

The Knewton Alta statistics course uses the Open Stax book. I make available on Canvas the PDF for that OpenStax book. But most of the students don’t read it. I have this really funny anecdote that I think summarizes textbooks for students. I asked a group of stat students in face-to-face class, “When do you read the stats book?” And this girl (who felt really comfortable) said, “When the professor sucks!”

I think that sums up the feeling of lots of students. If they don’t understand how the professor is teaching, that’s when they go to the textbook.

Most students would rather watch a video, read a short piece in Canvas, and try the homework, as opposed to read a chapter in a math textbook and try to follow that. And some of the textbooks are written really well, but still students’ level of motivation for reading is pretty low. But with the videos and the short bite-sized Canvas pages, they can read something, do an example, watch a video, solve a problem. That speaks to how millennials want to learn, expect to learn, how they learn.

That’s what they expect. And so that’s how I try to set it up. Knewton Alta does a pretty good job with that and there are a lot of videos and other resources. And the instruction that I’ve seen is pretty bite-sized.

Room for Improvement

My one “sad face” comment is that when I do the assessment, especially in this statistics class, a lot of the questions are still static. They’re not algorithmically generated. There are only a few of each learning objective to pick from. And if wanted to let students redo it, or if you wanted to pool questions, it just makes it a little harder. And so that is the one thing that is easier in MML; I can just grab a question from a learning objective, see the little red numbers that I know that are going to change for every single student, and I have less of a worry about academic integrity, students Googling stuff, that kind of thing. And in an age where I can do much less proctoring with my online students, that becomes a little bit bigger of a concern. It is the one thing that would make my life easier. Literally the only thing that they don’t do.