Thursday, September 13, 2012

Should Learners Design Their Own Assessments?

Assessment is an oft maligned, misunderstood, and under-appreciated component of instruction, which makes it a perfect vehicle for evangelizing the value of instructional design. Think about it. If we can get instructors and students excited about assessment… if we can change perceptions of assessment from “a dreaded endpoint” and “a necessary evil,” (actual learner quotes), or “the worst part of instructional design,” (actual ID quote), we can do… a lot of other cool stuff!

But, how? That’s the question kicking around one of my instructional design circles, followed closely by the suggestion to include learners in the assessment development process. What do you think? Should learners help determine what their assessment will look like, what it will measure, and what the measurement will ultimately mean?

As a corporate writer/instructional designer, I am a strong proponent of collaborative development and of performance-based assessment. Here are three reasons why I encourage learner participation in the assessment development process.

  1. Much of my professional work is designing training programs for pharmaceutical sales representatives. Theirs is a performance-based occupation, and professional assessments have to reflect and support that. Musial & Nieminen (2008) describe assessment as “the art of placing learners in a context that brings out or clarifies what a learner knows and can do, as well as what a learner may not know or cannot do.” I believe that a learner knows, better than anyone else, the reality of their performance context. While a manager or supervisor may establish minimum standards of performance, only the representatives themselves really know how the context of their jobs change from day to day.

  1. I gravitate toward performance assessment as described by Fuchs: “Three key features of performance assessment are: (1) students construct, rather than select, responses; (2) assessment formats allow teachers to observe student behavior on tasks reflecting real-world requirements; and (3) scoring reveals patterns in students’ learning and thinking,” (Fuchs, 1995), which again, makes self-evident the need to include learners in the development of their assessments.

  1. Taylor and Lamoreaux “point out that for the brain to make meaningful connections, learning needs to be tied to physical, embodied experience: ‘The brain’s physical responses to the sensory data are recorded—literally, embodied— as experience, hence accessible to reconstruction as memory; without such physical responses, there is no basis for constructing meaning’,” (cited in Merriam, 2008). If it is the student’s learning that is to be assessed, then it is the student’s experience that is to be reconstructed, and who is better qualified than the student to participate in this process?

With all that being said, I recognize there are times when assessment should be designed without learner input. For example, when the task is to memorize a set of fixed definitions (e.g., anatomy, grammar, or accounting) or when the student is incapable of participating in the development process (e.g., a young child or someone with intellectual limitations).

At the end of the day, it is incumbent on the instructional designer to make the best choices regarding assessment, considering the interrelationship of the instructional objectives and characteristics of the learner, the environment, and the instructor.

Sally Bacchetta

Fuchs, L. S. (1995). Connecting performance assessment to instruction: A comparison of behavioral assessment, mastery learning, curriculum-based measurement, and performance assessment. Retrieved from  (ERIC Digest No. E530).

Merriam, Sharan, B. (2008) Adult learning theory for the twenty-first century. New Directions for Adult & Continuing Education, 119 (pp. 93-98). 

Musial, D., Thomas, J., & Nieminen, G. (2008). Foundations of meaningful educational assessment. New York: McGraw-Hill. Chapter 1, "The Nature of Assessment" (pp. 3–22).

Saturday, September 8, 2012

To ADDIE or not to ADDIE?

I spent my earliest days as an instructional designer blissfully unaware of ADDIE or any other systems approach to ID. I asked questions, listened closely to the answers, thought on my feet and found my way, never missing the models I didn’t know existed. It’s not surprising, then, that I’ve never fallen in love with instructional design models the way some other designers have. In The Attack on ISD, Gordon & Zemke (2006) suggest that the heyday of the systems approach to instructional design has indeed passed, which led me to ponder these questions
  • What are the benefits of following ADDIE or any other ID model?
  •  Is there room for an instructional designer’s creativity and free thought when using an ID model?
  •  What role should ID models and instructional theory play in the daily work of an instructional designer?   
What are the benefits of following ADDIE or any other ID model?
I find ADDIE and other ID models most useful for framing the multitude of tasks involved in an instructional project, especially when working with people who lack a clear understanding of instructional design and the role of an ID. Outlining the main tenets of a model helps to clarify the overarching purpose and importance of instructional design, and introducing more specific detail as the project evolves demonstrates the validity and distinct function of instructional design.

Is there room for an instructional designer’s creativity when using an ID model?
There is if the designer has the motivation, skill, and confidence to take responsibility. Ultimately, it is incumbent on the instructional designer to ensure the fit between the instruction and the learners. “The professional challenge lies in the selection of the appropriate model or portions thereof that will be the best fit for the trainer, the training environment, the audience, and the content to be delivered,” (Cowell, Hopkins, McWhorter, & Jorden, 2006). In much the same way, physicians and other healthcare providers apply a standardized protocol to every patient contact, but the specific actions they select within that protocol are based on their determination of the best course of action, considering the interrelationship between the patient, the provider, the circumstance, the environment, and other relevant factors.

What role should ID models and instructional theory play in the daily work
of an instructional designer?
They should inform the designer’s approach to a project and facilitate the designer’s efforts to deliver the “right” instruction for the unique interrelationship between the learner, the environment, and the instructional objectives. “The professional trainer has the opportunity and the responsibility to select a model appropriate to the organization and learning needs of the audience for which the program is directed. In doing this it is common for professional trainers to select and meld those portions of various models that best fit their situations,” (Cowell, Hopkins, McWhorter, & Jorden, 2006).

One of the most important things I have learned as an ID is that models and theories are meant to be tools; the instructional designer is the artisan. The minute a designer surrenders control to their tools, the project begins to fail. I agree with Gordon & Zemke (2000) that “the harder you try to specify exactly what the designer must do in order to be ‘doing ISD’ the further into the wilderness you wander. That way lies madness.”

Cowell, C., Hopkins, P.C., McWhorter, R., & Jorden, D.L. (2006). Alternative training models. Advances in Developing Human Resources, 8(4), 460-475.

Gordon, J., & Zemke, R. (2000). The attack on ISD. Training, 37(4), 42-53