As an instructional designer, I find I frequently educate my organization on what I can accomplish and what they can accomplish with different strategies. Much (if not most) of my work is often not instructional. This is not common for instructional designers. I’ve embraced the fact since it gives me the opportunity to do something different and work with other teams. For example, I create a lot of presentations for leaders and write a lot of reference documentation (often tech writers and ISDs are considered interchangeable which isn’t necessarily true but I can see how people think that). I’ve even thought about referring to myself as an information designer since it is more accurate for the big picture of what I do and gives me the opportunity to expand my reach in business.
There are some really frustrating aspects to organizations not differentiating between instruction and information. For example, I am frequently asked to take information and create a reference. I have no problem with developing something that is not instructional because a lot of the same principles and techniques can be applied beautifully. I try to always explain when something is not instructional and why without long lectures but I see eyes glaze over and I think, “Right, I’m geeking out on them and I don’t blame them for just wanting me to just get on with it already.” However, it rarely fails that they will come back and then want to assess the reference. This generally leads to some head-banging on my part and careful conversations as I try to help them understand why assessing a reference is not an idea that “works.” I will ask them a series of questions:
- What is the objective of the assessment?
- What was the objective of the development?
Most of the time I hear that they want to ensure the recipients “understand” the reference. This is a tough one because now I have to show them that you cannot measure understanding. It is too subjective of a term. There are a lot of things that can be assessed where logically the inference is understanding but it will come down to “show me you can do what you are supposed to do when required.” That ends up being a complex, interactive assessment. It will be a great assessment that does the best job of proving behaviors back on the job but it also is all about instruction and not reference.
If I have verified that the client truly wanted a reference, the assessment is probably going to be about assessing that the recipients use it correctly when they necessary. This makes a lot of sense and I actually like these because we are assessing the use of the tool and not trying to get folks to memorize a ton of stuff they would find in the reference anyway. These assessments can be pretty simple by just offering a number of scenarios when it would be appropriate for them to use the reference tool and ask them what information they access and where.
This whole wind up is leading to a specific scenario that makes me cringe. I was developing a very dry, technical reference for a client and I heard someone on their team promise their leadership that there would be 10 questions written for this reference. This is like nails on a chalkboard for me and I have to really control the eye-roll because what they said makes no sense.
First of all, this is a reference, what are you assessing? What do you need to measure?
Secondly, there are no learning objectives for this reference so where are you coming up with the number 10? People love those nice round numbers but an assessment should be driven by the objectives and content, not by a nice round number. What if there are three learning objectives and to rigorously measure those objectives we only have seven questions? Or maybe we need 17 questions to measure the objectives?
I’ll tell you where they got the 10 question test idea; Kindergarten. (Well, probably it was elementary school but that doesn’t sound as good, does it?) In school we had tests that consisted of nice round numbers (probably easier for figuring out grades) and that sticks with people for the rest of their lives. This is the model of education most people know unless you are an adult education geek and boy, it can be really hard to get people to let go of that model of education. If it was good enough for them back in the day it should be good enough now! Heaven help me.