Friday, September 3, 2010

21st Century Tests

By now, you've likely heard that that the United States Department of Education announced awards to two separate coalitions of states to develop innovative large-scale educational assessments. What does it mean for a test to be innovative?

We are all used to multiple choice assessment tests. The primary reason that large scale assessments tend to use multiple choice is because they have always been far easier to score than open response tests.

However, is it possible that technology has finally enabled us to move beyond multiple choice tests? It certainly seems as if this is the case. Technology now has the ability to scan for specific words or phrases in student writing. Educators and educational publishers alike can use these scanning procedures to evaluate student writing.

Writing is obviously not the only skill that students should learn in school. If they are learning math in a high quality way, they should learn to apply mathematical skills to real world events. For example, they might use algebra and geometry to design structures. Virtual reality could evaluate whether or not students have the necessary math skills to design these structures. Given technological advancements it would also be fairly simple to evaluate the steps that a student completes as he/she designs the bridge. Partial points could easily be awarded.

It is time for technologists and educators to come together and develop large-scale assessments that demand critical thinking skills and are not solely based on multiple choice.

I'd love to discuss these ideas further with anybody interested in dong so.


  1. Hi Andrew. We are not there yet. We are just at the beginning of getting machines to understand natural language to discern "intent" not just syntax. Perhaps in the next 10 years, machine learning will advance enough that it was also be able to understand "intent" and "meaning." These domains are still in the art of teaching versus the science of teaching. It will take more than keyword searches to determine whether more caught than taught.

    While computers understand Math very well...until recently, it's actually been quite difficult to write a math equation that a human (read not a programmer) and machine can understand. We may actually achieve your vision in Mathematics long before we do in Language Arts.


  2. Cameron,

    Thanks for your comment.

    I think the challenging part for educators is to begin to think beyond the traditional. Many think that multiple choice tests are the only kinds of large scale standardized tests that exist. Consequently, they don't think beyond multiple choice in a serious way. As educational innovators we can begin to challenge this mentality. While technology may not yet be at the point where it can discern intent there are many different types of assessments that we can develop with technology that does exist - assessments that are both more authentic and more engaging than multiple choice.

    If a client wants me to develop multiple choice assessment questions I would be happy to do so. Because my expenses require me to be realistic. However, educational development requires a bit of idealism to be infused into the mix. Right now technologists are light years ahead of educators. Perhaps we can move towards a system in which both educators and technologists are driving one another towards meaningful advances.