I’ve given a lot of training sessions in my 20+ years of supporting The Examiner. Some folks are right on top of things, others are trainable, others just seem to spin their wheels and get nowhere fast. The wheel-spinners all seem to have one main thing in common: they get so involved in the process of assessment that they forget the product of the assessment is what they are really interested in.
What do I mean by “product of an assessment”? In my book that’s a couple of things:
- Finding out what an examinee knows, and what they don’t know.
- Seeing if the items in the assessment are valid.
- Seeing if the assessment itself is valid.
I’ll often tell folks that it doesn’t matter how they get to the results. Heck, they could give tests on an Ouija board if they get valid result. But it is the results that count. The problem is that people get so wrapped up with the process of the getting to the results that they spend all their time worrying about that and not what they really need to concern themselves with.
What are some symptoms of “process over product”?
- Someone is always saying “that’s not the way we’ve done it before”.
- People are designing test items and test designs before asking themselves what they want to find out.
- Committees spend all their time selecting a software product and leave little time for assessment development.
Schools (recipients of no-child-left-behind and state mandates) and businesses (we need to certify our employees!) both get hung up on the administrative side of assessments when they should be paying attention to the educational and training side. Yes, you need to worry about the assessment platform. But, you shouldn’t even think about that until you have it absolutely clear in your mind what you want to get out of the system.
With that in mind, a few tips:
- Before you do anything else, write down the goals. This shouldn’t be “I want the assessment to…” kind of things. It should be “I want the examinee to know…” or “The student should be able to…” objective statements.
- Figure out what things go into meeting these goals (yes… do a task analysis!).
- Determine the best method (not product!) for achieving this. Is it an assessment? An on-the-job evaluation? An interview?
- Then, and only then, figure out a “thing” for meeting the need.
I’d be remiss if I didn’t suggest that the Examiner Express (or our full Examiner System) would meet these needs. However, we aren’t the be-all and end-all of the assessment process. Go through the steps and take a good look at your process. Only then should you pick the methodology that meets your needs.
Tags: assessments, process, testing