This month Jose discusses standardized tests. Can big data end the “teach to the test” damage done by high stakes state assessments?

- The Knerds

It’s Not “Big Data vs. Teachers” —
It’s “Big Data vs. Standardized Tests”

by Jose Ferreira, Knewton founder and CEO
People often think that big data for education is a new thing. And it’s true that using big data the way Knewton does sounds almost like science fiction — our engine passively “norms” content at scale, uses normed content to determine students’ conceptual proficiencies to the percentile, and makes granular content recommendations for each student based on the combined anonymized data of all the other students in our network.

But there is an older, bricks-and-mortar type of big data for education: standardized tests.

People always ask me whether I think standardized tests actually measure anything. I used to give nerdy statistical answers about scoring validity and such. Eventually I realized that all they want to hear is that standardized tests are useless, possibly corrupt.

Standardized tests are increasingly unpopular, and not without reason But when used properly they do, or at least should, serve important purposes. There are two main kinds of standardized tests: admissions and state assessment. Admissions tests like the SAT were built to predict academic performance in college and graduate school. Grades and transcripts also do this, but academic standards and programs differ so greatly from school to school and region to region that a central standard measure is extremely useful. State assessments help demonstrate whether students are graduating from high school with basic literacy and math skills. They exist because society, which pays for every child to have free K-12 education, has a right to know that kids are actually learning.

To fill its only purpose (and to have a shot at being fair), a standardized test must yield totally consistent scores across administrations. If Maria takes the SAT in May, takes the summer off (and, let’s assume, doesn’t gain or lose any knowledge), and then takes the SAT again in September — she should get the exact same score both times (plus or minus the margin of error).

It is a nearly impossible technical challenge to build a test so well that it yields totally consistent scores across multiple examinations. Try holding a dinner party and giving every guest a 50-question test on any topic. Then invite them back the following week and give them another test on the same topic, but with different questions. Spare your friends the ordeal and trust me: most of them won’t receive similar scores. Yet the big U.S. test makers have figured out how to do exactly this, except they can do it repeatedly, at much larger scale, and with examinees about whom they know almost nothing.

The reality is that standardized tests are effective predictors of academic success in college or graduate school. That’s a fact of statistics, whether anyone likes it or not. It’s also just an average, and anyone could be an exception one way or the other. Standardized test scores, in my opinion, are more reliable at the top end of the scale than they are lower down. High scores generally indicate that someone is quite capable; but low scores may not mean very much at all. For these reasons, among others, standardized tests should be just part of the admissions process, and should never be the most important part.

The problem isn’t with the tests themselves, or their goals. The problems with major standardized tests are nearly always in how they are used. Read more ›
In the News
Knewton Partners with Elsevier to Build Adaptive Learning Products for Nursing, Health Professions, and Medical Students Globally...
 Read more ›
Education Reimagined: The Current Future of the Classroom...
 Read more ›
Buzzfeed examined our major new partnership with publishing giant Elsevier, which was also picked up by The Wall Street Journal...
 Read more ›
Education in Brazil & a Growing Middle Class
As the fifth most populous country in the world, Brazil has a population of approx. 200 million and one of the largest economies in the world. What does the educational system look like now, what’s in store for the future, and how will these developments drive economic growth? Read more ›
New Knerds
Welcome Emily, Max, Santosh, and Sandro (not pictured) ›
Movers and Shakers
Patrick Fogarty is an Educational Technology Consultant and the Instructional Technology director at New York's first 1:1 iPad school. The author of Going One-to-One: iPads and Mobile Devices in Education, Forgarty is also an Apple Distinguished Educator and puts together a great weekly digest of ed tech news.
“The teacher is the killer app in education.”
-John Martin, CEO of Sanoma Learning (overheard at EdTech Europe)
Knerd Alert
Front End Engineer Manager of Finance Operations
Software Engineer in Test Head of Human Resources
Senior Engineer - Implementation Tools Multilingual Partnership Manager
Mid Software Engineer Technical Recruiter
System Administrator Senior Producer (Marketing)
Adaptive Instruction Analyst

See all open jobs ›
ISTE 2014
The conference runs through July 1.
11th Annual Sloan Consortium Blended Learning Conference and Workshop
The conference runs from July 8-9.
The Knewton Symposium
“Designing the Future of Digital Education” runs from July 20-22.
Ed Tech 101
Metacognition is defined as knowledge about cognition or “knowing about knowing.” Read more ›
Ed Tech Buzz
How to Get Girls into Coding (The New York Times)

How Bill Gates Pulled Off the Swift Common Core Revolution (The Washington Post)

Starbucks Will Send Thousands of Employees to Arizona State for Degrees (The Chronicle of Higher Education)

Ed Tweets of the Month
Introducing the Interns, Part 1
Our first few summer interns arrived earlier this month. Read more ›
Education & Skills 2.0
Download Education & Skills 2.0: New Targets and Innovative Approaches, an e-book from the World Economic Forum, co-authored by Jose Ferreira, Knewton founder & CEO.
Up for Debate
The Case for Banning Laptops in the Classroom, a recent New Yorker article from Dartmouth computer science professor Dan Rockmore, has spurred debate not only on “electronic etiquette” but also on the way students currently use today's technology (and how they should). Slate and The Chronicle of Higher Education both offered responses. | Contact us | Subscribe