The last few weeks have not been good ones for some online test providers. Education Week reports that Indiana, Kentucky, Minnesota, and Oklahoma experienced significant technical problems with online testing:
Thousands of students experienced slow loading times of test questions, students were closed out of testing in mid-answer, and some were unable to log in to the tests. Hundreds, if not thousands, of tests may be invalidated.
In the comments to the article, Ze’ev Wurman observes:
The problem with scaling and testing for scaled-up customer base is far from trivial, and only few companies have mastered it reasonably well (Google, Yahoo, eBay, Amazon). Rolling out massive scaled up systems for millions of users with little to no tolerance for failure within the narrow testing window is — in my opinion — much beyond what PARC and SBAC are capable of doing.
In a follow up post, Education Week’s Catherine Gewertz notes:
None of this is cheery news for the two federally funded assessment consortia, whose tests live or die on states’ ability to administer online tests to all students in 2015. Spokesmen for those two groups offered publicly confident faces in response to last week’s testing disasters—the advantages of the new tests will outweigh the challenges; we’ll work out all the kinks before they roll out in 2015. But you don’t need to stretch too far to imagine that news of those disasters was not at all what the consortia needed to hear, as they seek to build confidence in the testing systems that more than 40 states are banking on.
Not only banking on, but attaching high-stakes to as well. What happens to a district’s participation rate for AYP purposes if hundreds or thousands of tests are invalidated? What happens to standardized test based school rankings and teacher evaluations if hundreds or thousands of tests are invalidated?
As for working out all the kinks, as the CCSSI Mathematics blog notes “Most logistics concerns over the new assessments seem to focus on upgrades in bandwidth or hardware, but it’s ultimately the software that will induce testing nightmares.” Click on the link to see a few software issues they identified in both PARCC and SBAC questions–and remember that the Smarter Balanced test bank will contain more than 10,000 items and tasks, each, as CCSSI Mathematics further observes, “with its own interface and underlying code, any single question of which can stymie students or crash the whole system during an assessment.”