The Knewton Blog

Our monthly newsletter features edtech and product updates, with a healthy dose of fun Knerd news.

How I Ensure Quality at Knewton

Posted in Knerds on April 27, 2011 by

Hi there. I’m Daniel and I’m the Quality Assurance (QA) Lead here at Knewton. When I was approached to write this blog post I said, “I am but a humble QA man… what would I write about?” But then I started to think about what I do here every day and thought that it might be interesting for you to understand what kind of quality controls our products go through to ensure they are the best for you, our customers.

I’ve been here a little over a year now, and as the head of QA I can tell you we take quality VERY seriously. All of are products go through several cycles of testing, with different members of our staff looking for different things (some look at the content, some at the user interface, some at the data, etc.). For example, as the QA Lead, it’s my job to organize how the testing will be done and manage the issues that are found. It’s a pretty daunting task when you have tight timelines.

Part of what we do is manual testing. In other words, someone sits in front a computer and steps through the system, looking for problems. Manual testing is important, but often it’s not enough. That’s why one of my main responsibility is automation. We’ve built several automation frameworks. We have unit tests written by our dev team that test our infrastructure and a front end automation framework around Selenium and Selenium 2. Selenium is an open source automation tool (open source software is designed to be used by anyone; users can download and modify the code as they wish). Selenium allows you to write code that automatically runs tests against your website in any browser that you choose (e.g. Firefox, Internet Explorer, Chrome).

Having this automation framework allows for several improvements in our QA process:

  • It makes it much faster to test new applications or new features.
  • It makes it harder to introduce new bugs without us knowing.
  • It makes it easier to retest things frequently.
  • It allows us to get new features or bug fixes out to production more quickly.
  • It helps find bugs that might not be found by manual testing alone.
  • It allows the developers to test new pieces of code rather quickly.
  • It allows for much faster cross-browser testing.
  • It allows us to keep costs down as fewer people are needed for testing.
  • Tests can be run in parallel

There are, however, some drawbacks to automation as well:

  • Tests need to be written in a programming language so your QA people need to be able to code.
  • The tests need consistent maintenance.
  • It can be hard to get tests to work across all browsers all the time.
  • Test run in an actual browser can be slow.

Still, these drawbacks are small in comparison to the gains we achieve by having it.

So, how does all this testing affect our customers? Well, if a new feature is requested, a new product is needed, or we find a bug and need to fix it ASAP, this automation is crucial to allowing a fast turnaround while still ensuring we give you the best quality product possible.