I'm all for taking sides when it makes sense. That said, I usually find debates about online vs. traditional learning kind of pointless. Much like debates between work-at-home and traditional career moms - "best" and "right" are defined situationally, in context; there is no actual "best" or "right." Is eLearning as good as face-to-face instruction? Is it better? Is the learning as sticky? It depends on the situation.
Lorri Freifeld's Training Magazine feature Online vs. In-Class Success is refreshing because Freifeld recognizes the importance of context when designing instruction. Training isn't an either or proposition; it's a What is the best training solution for this specific constellation of learner characteristics, performance objectives and resources? proposition.
That is the fundamental question instructional design should answer correctly. Every time.
Sally Bacchetta
Onwords™ column
My Google Profile+
Perspectives on all that is relevant and exciting in the world of instructional design.
Showing posts with label online learning. Show all posts
Showing posts with label online learning. Show all posts
Thursday, October 2, 2014
Wednesday, September 18, 2013
Cheating and the Online Environment
Cheating is an ongoing concern and a reality in both face-to-face and online settings (Oosterhof, Conrad, & Ely, 2008). If it is more common online there are at least two possible explanations: one, the physical
separation inherent in online learning facilitates cheating among those
who are so inclined (Rowe, 2004); and two, online instructors may
assess students more frequently in attempts to validate student
performance (Oosterhof, Conrad, & Ely, 2008), and more frequent assessment means more frequent occasions to cheat.
The nature of cheating has changed from copying answers from a student at the next desk to deliberately “crashing” a timed test to gain more time, hacking into instructor accounts and previewing assessment questions, and even changing grades in online student records (Cizek, 2001). Our society's reliance on test scores has risen dramatically over the last decade – driven at least in part by technologies that enable the administration and scoring of broad-scale assessments quickly and cost-effectively - and with that has come a disturbing trend of educators cheating and enabling or encouraging students to cheat (Cizek, 2001). In this climate, cheating (for some) has become a form of political or social protest: “Generally, there appears to be a growing indifference on the part of educators toward the behavior and even an increasing sense that cheating is a justifiable response to externally-mandated tests” (Cizek, 2001, p. 17).
The steps to be taken to minimize cheating in an online environment depend on the nature of and motivation for the cheating. For the purposes of this discussion, I offer three “global” strategies: creativity, judgments, and software.
Creativity in instructional design can reduce the frequency of cheating. For example, it is more difficult to cheat on a group project, performance assessment or essay test than on a multiple-choice quiz (Rowe, 2004). Designers can also “build in” security with frequent opportunities for assessment throughout the course or training. This can be particularly helpful in online learning where instructors don't have the benefits of face-to-face contact for assessing their students.
Humans are hard-wired to make judgments. Online instructors need to consider what they know about individual students (including their demonstrated abilities and the quality of their work) and group assessment norms and trends, and make judgments about whether cheating may have occurred. Also, instructors should consider the amount and type of work they assign. Some students report that they cheat because the workload is too heavy or the assignments are boring or meaningless (Stephans & Wengaard, 2001).
Software exists that can help increase testing security and minimize cheating. Assessment Systems Corporation is one company (I'm sure there are others) that offers test authoring, hosting, and psychometric services.
Should the definition of cheating evolve along with the tools we use to produce work in an online environment? A week ago, I would have answered, “No. Cheating is cheating is cheating and it's wrong.” But Maher (2008) has changed my thinking. “If a student is going to talk with a bunch of other students and network with them to exchange information to produce a paper, isn't that a skill that we want them to take to the workplace? If I can find someone who is working in advertising and who knows how to push a product, and they can collect information from other sources and borrow and steal and put it together and reshape it, isn't that a skill that I want them to have?” (Maher, 2008).
So perhaps the definition of cheating should evolve to fit with current notions of “work” and “learning”. But where, then, is the line between “collaboration” and “copying”? Maher has this to say: “... say that you're going to do something else that you can look at other people's projects, but the way I assess what you're doing is going to take into account that you're going to look at what other people are doing. Your work still has to be original, but to get inspiration from other people and to craft your work in response to theirs or alongside theirs is not something that's necessarily a problem.”I love that idea on its own, and I love it even more when I overlay Cizek (2001): “From the broadest perspective, it may be useful to entirely reconceptualize testing so that successful test performance can be more consistently and directly linked to student effort and effective instruction, and so that unsuccessful performance is accompanied by sufficient diagnostic information about students’ strengths and weaknesses” (p. 10).
Sally
References
Cizek, G. J. (2001). An Overview of Issues Concerning Cheating on Large-Scale Tests. Paper presented at the annual meeting of the National Council on Measurement in Education, April 2001, Seattle, WA.
Maher, S. (2008). Interviews. Retrieved from http://www.pbs.org/wgbh/pages/frontline/kidsonline/interviews/maher.html#5
Oosterhof, A., Conrad, R.-M., & Ely, D. P. (2008). Assessing learners online. Upper Saddle River, NJ: Pearson.
Rowe, N. (2004). Cheating in online student assessment: Beyond plagiarism. Online Journal of Distance Learning Administration, 7(2). Retrieved from http://www.westga.edu/~distance/ojdla/summer72/rowe72.html
Stephans, J. M., & Wangaard, D. B. (2001). Teaching for integrity: Steps to prevent cheating in your classroom. Retrieved from http://www.ethicsed.org/programs/integrity-works/pdf/teachingforintegrity.pdf.
The nature of cheating has changed from copying answers from a student at the next desk to deliberately “crashing” a timed test to gain more time, hacking into instructor accounts and previewing assessment questions, and even changing grades in online student records (Cizek, 2001). Our society's reliance on test scores has risen dramatically over the last decade – driven at least in part by technologies that enable the administration and scoring of broad-scale assessments quickly and cost-effectively - and with that has come a disturbing trend of educators cheating and enabling or encouraging students to cheat (Cizek, 2001). In this climate, cheating (for some) has become a form of political or social protest: “Generally, there appears to be a growing indifference on the part of educators toward the behavior and even an increasing sense that cheating is a justifiable response to externally-mandated tests” (Cizek, 2001, p. 17).
The steps to be taken to minimize cheating in an online environment depend on the nature of and motivation for the cheating. For the purposes of this discussion, I offer three “global” strategies: creativity, judgments, and software.
Creativity in instructional design can reduce the frequency of cheating. For example, it is more difficult to cheat on a group project, performance assessment or essay test than on a multiple-choice quiz (Rowe, 2004). Designers can also “build in” security with frequent opportunities for assessment throughout the course or training. This can be particularly helpful in online learning where instructors don't have the benefits of face-to-face contact for assessing their students.
Humans are hard-wired to make judgments. Online instructors need to consider what they know about individual students (including their demonstrated abilities and the quality of their work) and group assessment norms and trends, and make judgments about whether cheating may have occurred. Also, instructors should consider the amount and type of work they assign. Some students report that they cheat because the workload is too heavy or the assignments are boring or meaningless (Stephans & Wengaard, 2001).
Software exists that can help increase testing security and minimize cheating. Assessment Systems Corporation is one company (I'm sure there are others) that offers test authoring, hosting, and psychometric services.
Should the definition of cheating evolve along with the tools we use to produce work in an online environment? A week ago, I would have answered, “No. Cheating is cheating is cheating and it's wrong.” But Maher (2008) has changed my thinking. “If a student is going to talk with a bunch of other students and network with them to exchange information to produce a paper, isn't that a skill that we want them to take to the workplace? If I can find someone who is working in advertising and who knows how to push a product, and they can collect information from other sources and borrow and steal and put it together and reshape it, isn't that a skill that I want them to have?” (Maher, 2008).
So perhaps the definition of cheating should evolve to fit with current notions of “work” and “learning”. But where, then, is the line between “collaboration” and “copying”? Maher has this to say: “... say that you're going to do something else that you can look at other people's projects, but the way I assess what you're doing is going to take into account that you're going to look at what other people are doing. Your work still has to be original, but to get inspiration from other people and to craft your work in response to theirs or alongside theirs is not something that's necessarily a problem.”I love that idea on its own, and I love it even more when I overlay Cizek (2001): “From the broadest perspective, it may be useful to entirely reconceptualize testing so that successful test performance can be more consistently and directly linked to student effort and effective instruction, and so that unsuccessful performance is accompanied by sufficient diagnostic information about students’ strengths and weaknesses” (p. 10).
Sally
References
Cizek, G. J. (2001). An Overview of Issues Concerning Cheating on Large-Scale Tests. Paper presented at the annual meeting of the National Council on Measurement in Education, April 2001, Seattle, WA.
Maher, S. (2008). Interviews. Retrieved from http://www.pbs.org/wgbh/pages/frontline/kidsonline/interviews/maher.html#5
Oosterhof, A., Conrad, R.-M., & Ely, D. P. (2008). Assessing learners online. Upper Saddle River, NJ: Pearson.
Rowe, N. (2004). Cheating in online student assessment: Beyond plagiarism. Online Journal of Distance Learning Administration, 7(2). Retrieved from http://www.westga.edu/~distance/ojdla/summer72/rowe72.html
Stephans, J. M., & Wangaard, D. B. (2001). Teaching for integrity: Steps to prevent cheating in your classroom. Retrieved from http://www.ethicsed.org/programs/integrity-works/pdf/teachingforintegrity.pdf.
Tuesday, August 6, 2013
Technology and Multimedia in Online Learning
Introduce a technology to an online learning experience, and you tee up the question, “What impact does this technology have on the online learner/learner group/facilitator?” Obvious answers are that technology:
- Enables students and facilitators to communicate conveniently across geographic boundaries<br>
- Facilitates the development of technical skills<br>
- Offers opportunities to enrich learning with interactive multimedia design elements
But do technology and multimedia actually change the value proposition of online learning? They can; whether or not they do is a function of instructional design and facilitation.
“The greatest challenge in assessing an online engaged activity is determining the quality of thought expressed” (Conrad & Donaldson, 2011, p. 29). This is a critical need, and technology can help here. The Discussion Analysis Tool “also known as ForumManager (Jeong, 2003) evaluates patterns in online interactions” (Conrad & Donaldson, 2011, p. 29). ForumManager analyzes the quantity and depth of discussion entries per participant. That's one tool. Instructional design is another. Aligning discussion board activities with Bloom's Taxonomy encourages critical thinking that can improve learner success and satisfaction.
As I (and others) have said before, the primary consideration to be made about implementing technology for online learning is whether or not the technology supports achievement of learning objectives. Cool whiz bang multimedia that does nothing to enrich learning is merely a fancy and costly distraction. Technology that requires too much of learners - either because the tech is too time-consuming or user-unfriendly, or because the learner group lacks requisite experience with the technology – undermines the very learning process it is supposed to support.<br>
The craft of instructional design is as fluid as technology itself, and will continue to evolve in response to learner need and emerging technology. Our charge is to make sure our decisions are driven by learner need, not by our interest in technology.<br>
References
Conrad, R. & Donaldson, J. (2011). Engaging the online learner; Activities and resources for creative instruction. San Francisco, CA: Jossey-Bass.
Jeong, A. (2003). Sequential analysis of group interaction and critical thinking in online threaded discussions. American Journal of Distance Education, 17(1), 25-43.
Friday, July 19, 2013
Setting up an Online Learning Experience
Starting well is essential for a successful online learning experience, and it doesn’t happen automatically. A skilled facilitator spends the time necessary to achieve social, cognitive, and teaching presence from the outset (Boettcher & Conrad, 2010) and continues to actively support learners throughout the course according to their individual needs and learning styles.
Having a comprehensive understanding of available technology enables the facilitator to select the best technology for a particular application and to effectively coach learners who have not mastered the technology. Without technological clarity, a facilitator may overload herself and the learners with unnecessary tasks or technology that interferes with learning (Boettcher & Conrad, 2010).
It is essential to clearly communicate expectations to learners. Most adult learners are goal-oriented, and mastery of skills boosts their confidence and improves their self-esteem (Malamed, 2013). Learners need to understand expectations so they can take steps to meet them and feel satisfied with their online learning experience (Boettcher & Conrad, 2010).
When setting up an online learning experience, facilitators should provide an opportunity for learners to share biographical information. This fosters openness and community, and also provides a frame of reference for the facilitator when interacting with learners in discussion forums. When a facilitator uses student names and references relevant biographical information, this personalizes the learning experience and promotes learner engagement (Laureate Education, Inc., n.d.).
Conversely, facilitators need to be aware that some learners are reluctant to share personal information for fear of being stereotyped (e.g., by race, gender, ethnicity, geography, relationship status, etc. (Laureate Education, Inc., n.d.). Personal information should be invited, not required. As well, when designing ice breakers and discussion prompts, facilitators should allow learners to choose from activities with varying degrees of openness.
Great beginnings lead to great middles and great endings!
Sally
References
Boettcher, J. V., & Conrad, R. (2010). The online teaching survival guide: Simple and practical pedagogical tips. San Francisco, CA: Jossey-Bass.
Laureate Education, Inc. (Producer) (n.d.) Launching the Online Learning Experience [DVD] Baltimore: MD.
Malamed, C. (2013). http://theelearningcoach.com/learning/characteristics-of-adult-learners
Having a comprehensive understanding of available technology enables the facilitator to select the best technology for a particular application and to effectively coach learners who have not mastered the technology. Without technological clarity, a facilitator may overload herself and the learners with unnecessary tasks or technology that interferes with learning (Boettcher & Conrad, 2010).
It is essential to clearly communicate expectations to learners. Most adult learners are goal-oriented, and mastery of skills boosts their confidence and improves their self-esteem (Malamed, 2013). Learners need to understand expectations so they can take steps to meet them and feel satisfied with their online learning experience (Boettcher & Conrad, 2010).
When setting up an online learning experience, facilitators should provide an opportunity for learners to share biographical information. This fosters openness and community, and also provides a frame of reference for the facilitator when interacting with learners in discussion forums. When a facilitator uses student names and references relevant biographical information, this personalizes the learning experience and promotes learner engagement (Laureate Education, Inc., n.d.).
Conversely, facilitators need to be aware that some learners are reluctant to share personal information for fear of being stereotyped (e.g., by race, gender, ethnicity, geography, relationship status, etc. (Laureate Education, Inc., n.d.). Personal information should be invited, not required. As well, when designing ice breakers and discussion prompts, facilitators should allow learners to choose from activities with varying degrees of openness.
Great beginnings lead to great middles and great endings!
Sally
References
Boettcher, J. V., & Conrad, R. (2010). The online teaching survival guide: Simple and practical pedagogical tips. San Francisco, CA: Jossey-Bass.
Laureate Education, Inc. (Producer) (n.d.) Launching the Online Learning Experience [DVD] Baltimore: MD.
Malamed, C. (2013). http://theelearningcoach.com/learning/characteristics-of-adult-learners
Wednesday, July 3, 2013
Online Learning Communities
How do online learning communities significantly impact both student learning and satisfaction within online courses? Compared to traditional education, online learning communities offer more opportunities for learner-to-learner engagement (Laureate, n.d.). Through online learning communities learners exchange ideas and information with more and further flung peers than is possible in traditional educational settings. This aggregation of multiple intelligence (Gardner, 2003) exposes learners to different learning styles, which can increase student satisfaction and both real and perceived learning (Gilbert & Han, 2002).
What are the essential elements of online community building? The goal on an online learning community is “a sense of co-created knowledge and meaning” (Laureate, n.d.). Essential elements are those that support that goal, including:
• Orientation to online learning and to the course
• Navigation that is intuitive and clear for learners with varying degrees of technical ability
• Presence of the facilitator or instructor early and often
• Invitation to students to post a bio of themselves
• Feedback that is timely, specific, and supportive (Laureate, n.d.).
How can online learning communities be sustained? The long-term health of an online learning community depends on robust and ongoing co-creation of learning experiences in which learners and facilitators participate and evolve to the fullness of their abilities (Laureate, n.d.).
What is the relationship between community building and effective online instruction? Having been a member of an online learning community for 14 months, I know that community is a prerequisite for effective online instruction. You don't have to take my word for it: “Knowledge is literally the set of connections between entities, or the adjustment of the strengths of those connections,” (Downes, 2012) and “Experiences with the environment are critical to learning,” (Ertmer & Newby, 1993).
What did you learn that will help you become a more effective instructor in the future? I learn most about this by example, and I will seek to replicate the successes of some of my online professors who are:
• Organized – Online learning is fast-paced and intense. Instructors with excellent time and project management skills afford their students the best opportunities to interact with information and each other and thus, make deeper connections with the course material.
• Present – Online instructors who establish social presence in the community set a tone of openness and collaboration. Instructors who are present are able to probe for understanding and challenge learners to reflect deeply on course material, so they can enrich the group discussions.
• Clear – Instructors who give clear direction enable students to deliver as expected and experience mastery of course material.
Sally Bacchetta
References
Downes, S. (2012). Connectivism and Connective Knowledge: Essays on meaning and learning networks.
Ertmer, P. and Newby, T. (1993). Behaviorism, Cognitivism, Constructivism: Comparing Critical Features from an Instructional Design Perspective. Performance Improvement Quarterly, 6(4), pp. 50-72.
Gardner, H. (2003, April 21). Multiple intelligences after 20 years. Paper presented to the American Educational Research Association, Chicago, IL. Retrieved from http://www.pz.harvard.edu/Pls/HG_MI_after_20_years.pdf.
Gilbert, J.E. & Han, C.Y. (2002). Arthur: A personalized instructional system. Journal of Network and Computing Applications, 22(3), 149-160.
Laureate Education, Inc. (Producer). (n.d.). Online Learning Communities [DVD] Baltimore, MD
What are the essential elements of online community building? The goal on an online learning community is “a sense of co-created knowledge and meaning” (Laureate, n.d.). Essential elements are those that support that goal, including:
• Orientation to online learning and to the course
• Navigation that is intuitive and clear for learners with varying degrees of technical ability
• Presence of the facilitator or instructor early and often
• Invitation to students to post a bio of themselves
• Feedback that is timely, specific, and supportive (Laureate, n.d.).
How can online learning communities be sustained? The long-term health of an online learning community depends on robust and ongoing co-creation of learning experiences in which learners and facilitators participate and evolve to the fullness of their abilities (Laureate, n.d.).
What is the relationship between community building and effective online instruction? Having been a member of an online learning community for 14 months, I know that community is a prerequisite for effective online instruction. You don't have to take my word for it: “Knowledge is literally the set of connections between entities, or the adjustment of the strengths of those connections,” (Downes, 2012) and “Experiences with the environment are critical to learning,” (Ertmer & Newby, 1993).
What did you learn that will help you become a more effective instructor in the future? I learn most about this by example, and I will seek to replicate the successes of some of my online professors who are:
• Organized – Online learning is fast-paced and intense. Instructors with excellent time and project management skills afford their students the best opportunities to interact with information and each other and thus, make deeper connections with the course material.
• Present – Online instructors who establish social presence in the community set a tone of openness and collaboration. Instructors who are present are able to probe for understanding and challenge learners to reflect deeply on course material, so they can enrich the group discussions.
• Clear – Instructors who give clear direction enable students to deliver as expected and experience mastery of course material.
Sally Bacchetta
References
Downes, S. (2012). Connectivism and Connective Knowledge: Essays on meaning and learning networks.
Ertmer, P. and Newby, T. (1993). Behaviorism, Cognitivism, Constructivism: Comparing Critical Features from an Instructional Design Perspective. Performance Improvement Quarterly, 6(4), pp. 50-72.
Gardner, H. (2003, April 21). Multiple intelligences after 20 years. Paper presented to the American Educational Research Association, Chicago, IL. Retrieved from http://www.pz.harvard.edu/Pls/HG_MI_after_20_years.pdf.
Gilbert, J.E. & Han, C.Y. (2002). Arthur: A personalized instructional system. Journal of Network and Computing Applications, 22(3), 149-160.
Laureate Education, Inc. (Producer). (n.d.). Online Learning Communities [DVD] Baltimore, MD
Monday, July 1, 2013
Continuing the Journey
Today I move into the next leg of my journey toward a Master's degree in Instructional Design with a course called Online Instructional Strategies. I'm glad you've come along. I'm looking forward to more growing and learning together. Welcome!
Sally Bacchetta
Sally Bacchetta
Insights on Program Evaluation
Having just completed a Program Evaluation course at Walden University, I reflect here on the experience.
I didn’t realize that the process of creating and presenting my program evaluation plan was central to my learning until it was done. It was in pulling the disparate pieces of the plan together into a cohesive whole that I found meaning. Only then did I realize that had I not thoroughly explored the program context; not stretched myself to identify primary and secondary stakeholders and their interests, needs, and biases; not named my values and committed myself to them; not reflected on my own biases; not considered the impact of my report, my evaluation plan, which may have held together well enough to bring me to the last week of the course, would ultimately have failed the final analysis.
Program evaluation can inform what needs to change and form the basis of a change management plan, but first, it must fit the program context. It is incumbent on the evaluator to consider how contextual factors may inform the selection of an evaluation model. Program evaluation is inherently a political process, and an evaluator who ignores, avoids, or mismanages the political realities of evaluation limits the effectiveness and usefulness of the process (Fitzpatrick, Sanders, & Worthen, 2010.
This experience has shown me that working with stakeholders is one of the most challenging aspects of program evaluation. The important work of planning a program evaluation can be upset by stakeholder conflict, politics, bias, and unexpected manifestations of organizational culture. And yet, as an evaluator, I have a professional obligation to find my way to promote meaningful evaluation and the application of evaluation results by stakeholders (Fitzpatrick et al., 2010).
Technology can facilitate communication with stakeholders and simplify the processes of data collection, data management, and research (Laureate Education, Inc., n.d.), but it is only a tool; the evaluator must provide the raw material and craft the work. In every phase of evaluation it is incumbent on the evaluator to uphold the priority of justice (Schweigert, 2007); to mine the program context for cultural cues, gaps in understanding, potential bias; and feasibility; and to demonstrate and promote respect for stakeholders and the evaluation process. Bias is the weed that pervades the evaluation process, from the evaluator’s preference for a particular approach or data collection design to overt or covert liking of some stakeholders more than others, finding some steps of the evaluation process more interesting, more compelling, or more exhausting than others. The presence of bias is a given. Evaluators must be frankly self-reflective about their role in the evaluation process and circumspect about client requests, so as to minimize the potential for bias and ethical compromise (Fitzpatrick et al., 2010).
At some point, situational circumstance requires evaluators to make interpretations and best guesses (Schweigert, 2007), which are subject to bias and ethical compromise. I carry with me from this course Sieber’s (1980) conclusion that “being ethical in program evaluation is a process of growth in understanding, perception, and creative problem-solving ability that respects the interests of individuals and of society” (p. 53).
Sally Bacchetta
References
American Evaluation Association, 2004. Guiding principles. Retrieved from www.eval.org/Publications/Guiding Principles.asp.
Fetterman, D. (2001). The transformation of evaluation into a collaboration: A vision of evaluation in the 21st century. American Journal of Evaluation, 22(3), 381–384. Retrieved from the Education Research Complete database
Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Boston, MA: Pearson
Laureate Education, Inc. (Producer). (2009). Formative and Summative Evaluation. [DVD] United States.
Schweigert, F. J. (2007). The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30(4), 394–399.
Sieber, J. E. (1980). Being ethical: Professional and personal decisions in program evaluation. In R.E. Perloff & E. Perloff (Eds.), Values, ethics, and standards in evaluation. New Directions for Program Evaluation, No. 7, 51-61. San Francisco: Jossey-Bass.
Worthen, B. (2001). Whither evaluation? That all depends. American Journal of Evaluation, 22(3), 409–416. Retrieved from the Education Research Complete database
I didn’t realize that the process of creating and presenting my program evaluation plan was central to my learning until it was done. It was in pulling the disparate pieces of the plan together into a cohesive whole that I found meaning. Only then did I realize that had I not thoroughly explored the program context; not stretched myself to identify primary and secondary stakeholders and their interests, needs, and biases; not named my values and committed myself to them; not reflected on my own biases; not considered the impact of my report, my evaluation plan, which may have held together well enough to bring me to the last week of the course, would ultimately have failed the final analysis.
Program evaluation can inform what needs to change and form the basis of a change management plan, but first, it must fit the program context. It is incumbent on the evaluator to consider how contextual factors may inform the selection of an evaluation model. Program evaluation is inherently a political process, and an evaluator who ignores, avoids, or mismanages the political realities of evaluation limits the effectiveness and usefulness of the process (Fitzpatrick, Sanders, & Worthen, 2010.
This experience has shown me that working with stakeholders is one of the most challenging aspects of program evaluation. The important work of planning a program evaluation can be upset by stakeholder conflict, politics, bias, and unexpected manifestations of organizational culture. And yet, as an evaluator, I have a professional obligation to find my way to promote meaningful evaluation and the application of evaluation results by stakeholders (Fitzpatrick et al., 2010).
Technology can facilitate communication with stakeholders and simplify the processes of data collection, data management, and research (Laureate Education, Inc., n.d.), but it is only a tool; the evaluator must provide the raw material and craft the work. In every phase of evaluation it is incumbent on the evaluator to uphold the priority of justice (Schweigert, 2007); to mine the program context for cultural cues, gaps in understanding, potential bias; and feasibility; and to demonstrate and promote respect for stakeholders and the evaluation process. Bias is the weed that pervades the evaluation process, from the evaluator’s preference for a particular approach or data collection design to overt or covert liking of some stakeholders more than others, finding some steps of the evaluation process more interesting, more compelling, or more exhausting than others. The presence of bias is a given. Evaluators must be frankly self-reflective about their role in the evaluation process and circumspect about client requests, so as to minimize the potential for bias and ethical compromise (Fitzpatrick et al., 2010).
At some point, situational circumstance requires evaluators to make interpretations and best guesses (Schweigert, 2007), which are subject to bias and ethical compromise. I carry with me from this course Sieber’s (1980) conclusion that “being ethical in program evaluation is a process of growth in understanding, perception, and creative problem-solving ability that respects the interests of individuals and of society” (p. 53).
Sally Bacchetta
References
American Evaluation Association, 2004. Guiding principles. Retrieved from www.eval.org/Publications/Guiding Principles.asp.
Fetterman, D. (2001). The transformation of evaluation into a collaboration: A vision of evaluation in the 21st century. American Journal of Evaluation, 22(3), 381–384. Retrieved from the Education Research Complete database
Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Boston, MA: Pearson
Laureate Education, Inc. (Producer). (2009). Formative and Summative Evaluation. [DVD] United States.
Schweigert, F. J. (2007). The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30(4), 394–399.
Sieber, J. E. (1980). Being ethical: Professional and personal decisions in program evaluation. In R.E. Perloff & E. Perloff (Eds.), Values, ethics, and standards in evaluation. New Directions for Program Evaluation, No. 7, 51-61. San Francisco: Jossey-Bass.
Worthen, B. (2001). Whither evaluation? That all depends. American Journal of Evaluation, 22(3), 409–416. Retrieved from the Education Research Complete database
Wednesday, May 15, 2013
The Political Nature of Program Evaluation
Program evaluation is a
political process. An evaluator who ignores, avoids, or mismanages the
political realities of evaluation limits the effectiveness and usefulness of
the process (Fitzpatrick, Sanders, & Worthen, 2010). Ethical complexities wind in and
among the more overt political features of evaluation such as financial support,
stakeholder allegiance, and social impact. Morris and Cohn (1993) detail several
ways in which stakeholders may seek to influence evaluation outcomes, and Fitzpatrick
et al. (2010) caution that evaluators also need to be aware of their own
potential to taint the evaluative process.
If we accept that
evaluation is political (and, therefore, ripe for ethical complication), then
we must ask how best to balance the objectivity required in a program
evaluation with the political interests of stakeholders. We must ask, “What ethical
standards and values need to be emphasized in program evaluation?”
The American Evaluation
Association’s (AEA) Program Evaluation
Standards (Yarbrough, Shulha, Hopson, & Caruthers, 2011) and Guiding Principles for Evaluators (American
Evaluation Association, 2004) provide a broad, somewhat obvious, framework for
ethical conduct.
Fitzpatrick et al. (2010)
are more specific, encouraging evaluators to be both self-reflective about their
role in the evaluation process and circumspect about client requests, so as to
minimize the potential for bias and ethical compromise: “…the client may be
asking for what the client perceives as editing changes, but the evaluator sees
as watering down the clarity or strength of the judgments made” (p. 81).
And Schweigert (2007) roots
evaluator responsibility in the notion of justice – public, procedural, and
distributive.
From this we can extract answers to the question “What ethical standards and values need
to be emphasized in program evaluation?”
Ethical standards:
- Those detailed in the AEA’s and other professionally recognized codes of conduct.
Values:
- Commitment to truth – what Schweigert (2007) calls the priority of justice
- Cultural sensitivity
- Respect (for stakeholders, ourselves, and the evaluation process)
It seems that no professional
code nor personal charter can do the whole job. No matter how pointed the professional
standards, situational circumstance requires evaluators to make interpretations
and best guesses (Schweigert, 2007), which are subject to bias and ethical
compromise, as Weiss (2006) lays bare any illusions we may have that we are
above or beyond the snare of bias and ethical confusion: “You never start from
scratch. We pick up the ideas that are congenial to our own perspective.
Therefore, people pick up this thought or that interpretation of a research
report that fits with what they know or what they want to do” (p. 480).
I have thought about this a lot over the past few days, returning
again and again to Sieber’s (1980) conclusion that “being ethical in program evaluation is
a process of growth in understanding, perception, and creative problem-solving
ability that respects the interests of individuals and of society” (p. 53).
References
American Evaluation
Association, 2004. Guiding principles. Retrieved from www.eval.org/Publications/Guiding
Principles.asp.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program
evaluation: Alternative approaches and practical guidelines (4th ed.).
Boston, MA: Pearson
Morris, M., & Cohn, R. (1993). Program evaluators and
ethical challenges: A national survey. Evaluation
Review, 17, 621-642.
Schweigert,
F. J. (2007). The priority of justice: A framework approach to ethics in
program evaluation. Evaluation and Program Planning, 30(4), 394–399.
Sieber, J. E.
(1980). Being ethical: Professional and personal decisions in program
evaluation. In R.E. Perloff & E. Perloff (Eds.), Values, ethics, and standards in evaluation. New Directions for
Program Evaluation, No. 7, 51-61. San Francisco: Jossey-Bass.
Weiss, C. H., & Mark, M. M. (2006). The oral history of
evaluation Part IV: The professional evolution of Carol Weiss. American Journal of Evaluation, 27(4),
p. 474-483.
Thursday, May 2, 2013
Fostering Behavior Change
In Fostering Behavior Change (Tulgan, 2013), Bruce Tulgan offers seven best practices for
creating training that increases knowledge uptake and meaningful behavior
change. Two-thirds of my way through a Master’s degree in Instructional Design
and Technology, I first thought Tulgan’s tips were obvious. Simplistic. After
thinking about it quite a bit, I’m sure
they are. Why would Tulgan, an established training expert, tell us what we
already know? Because it’s true. Because he’s right.
There is no
magic to training, and all the cool whiz-bang technology in the world doesn’t
change the fact that effective training is a product of sound design and
delivery. Tulgan’s tips should seem
obvious, because he is reminding instructional designers and trainers of what
we already know, yet sometimes fail to execute. We need to leverage needs
assessments to align instructional objectives with identifiable skill and
knowledge gaps, link instructional content to real-life, and deliver content to
multiple memory centers. Sticky training offers actionable solutions and
learning extensions. Finally, we need to follow up and cultivate support for
ongoing learning.
We know this.
We need to do it. Every time.
Make a great day,
Reference
Tulgan, B.
(2013, January/February). Fostering behavior change. Training, 50(1), p. 9.
Tuesday, March 5, 2013
Project Management in Education and Training
I'm kicking off a new experience in my journey to a Master's degree in Instructional Design and Technology. This term I am taking a course called Project Management in Education and Training. I'm looking forward to expanding my understanding of how we make projects tick, and I welcome you along for the ride.
Make a great day,
Sally Bacchetta
Make a great day,
Sally Bacchetta
Sunday, January 20, 2013
Review of Canvas CMS
The
fields of instructional design and distance learning are in turbo-growth mode,
with new technology (and improvements to "old" technology)
barreling out from every direction. I've just recently begun exploring CMS
(Content Management Systems), also called LMS (Learning Management
Systems), and I've decided to share my very informal reviews here. First up,
Canvas (http://www.instructure.com).
I liked Canvas right away. The site is attractive, and the intuitive navigation enabled me to feel comfortable and competent almost immediately. Canvas cleanly blends Web 2.0 tools and other technologies into features that distance learners want and need, like:
Sally Bacchetta
I liked Canvas right away. The site is attractive, and the intuitive navigation enabled me to feel comfortable and competent almost immediately. Canvas cleanly blends Web 2.0 tools and other technologies into features that distance learners want and need, like:
- Mobility – “Users can receive Canvas notifications via Facebook, SMS text messaging, email and other communication channels” (http://www.instructure.com).
- Outcomes to Objective – “With Canvas, you can connect each learning action to a specific goal—which means results are demonstrated in clearly measurable ways” (http://www.instructure.com).
- Managed Hosting – “Canvas eliminates the time and expense of setting up, configuring and maintaining your own servers” (http://www.instructure.com)
- Analytics – Built-in tools help designers and instructors evaluate the effectiveness of their instruction and specific content, which I could use to demonstrate ROI and justify my design decisions to clients, and more importantly, make changes as necessary to achieve desired outcomes.
- Open Source – Software can be tweaked and freely shared with others.
Sally Bacchetta
Thursday, January 17, 2013
On Wedemeyer's Theory of Independent Study
Charles
Wedemeyer (1981) rooted his Theory of Independent Study in the ideal of
learner freedom. Wedemeyer characterized independent study as one in
which:
- The student and teacher are separated.
- The normal processes of teaching and learning are carried out in writing or through some other medium.
- Teaching is individualized.
- Learning takes place through the student’s activity.
- Learning is made convenient for the student in his or her own environment,
- The learner takes responsibility for the pace of his or her own progress, with freedom to start and stop at any time (Simonson, Smaldino, Albright, & Zvacek, 2012, p. 44).
Does Wedemeyer’s construct meet the criteria of a theory
of distance learning? Based on my examination of relevant literature I
believe it does. Effective theory guides our practice and renders
instruction more effective for the learner (Saba, 2009). If I use
Wedemeyer’s ideals as guidance, I will design instruction that is
individualized, fosters learner autonomy, and provides appropriate
support (teaching), all of which may make the instruction more effective
for the learner.
Garrison
(2000) categorized theory as something that “will explain and
anticipate distance education practices for a broad range of emerging
educational purposes and experiences” (Garrison, 2000). Wedemeyer’s
theory is fluid enough to have informed what was current practice at the
time as well as inform current practice today. Wedemeyer’s contribution
to the Articulated Instructional Media (AIM) project in 1964
demonstrated his theory in practice and the value of it to both explain
and anticipate emergent educational imperatives: “it was proposed that a
unique system be developed for a new type of institution . . . made
possible through course design utilizing media and technology and . . .
supported by counseling and resource and learning centres” (Sherow &
Wedemeyer, 1990, p. 18).
Is
Wedemeyer’s theory relevant to distance learning today? My research
review suggests that it is. His theory comprises four elements of
distance learning: teacher, learner or learners, method of
communication, and instructional content (Simonson, Smaldino, Albright,
& Zvacek, 2012), which closely resembles a currently accepted
definition of distance education: “institution-based, formal education
where the learning group is separated, and where interactive
telecommunications systems are used to connect learners, resources, and
instructors” (Schlosser & Simonson, 2009, p. 1).
Although
he emphasized the importance of learner autonomy and
self-responsibility, Wedemeyer also stressed the unique and important
role of the teacher in distance learning (Simonson, Smaldino, Albright,
& Zvacek, 2012). This distinguishes his theory from one of distance self-study
and demonstrates relevance for instructional design today, as the
preponderance of evidence underscores the importance of a teaching
presence in distance learning (Swan & Shih, 2005; Meyer, 2003;
Wu & Hiltz, 2004), and Garrison (2007) asserts a causal
relationship between teaching presence and “ student satisfaction,
perceived learning, and sense of community” (Garrison, 2007, p. 67).
References
Garrison, D. R. (2007). Online community of inquiry review: Social, cognitive, and teaching
presence issues. Journal of Asynchronous Learning Networks 11(1). pp. 61-72.
Garrison, R. (2000). Theoretical challenges for distance education in the 21st century: A shift
from structural to transactional issues. The International Review of Research in Open and
Distance Learning.
Laureate Education, Inc. (Producer). (2009). Theory and distance learning [DVD]. Baltimore,
MD: Author
Meyer, K. A. (2003). Face-to-face versus threaded discussions: The role of time and higher-
order thinking. Journal of Asynchronous Learning Networks 7 (3), p. 55-65.
Schlosser, L., & Simonson, M. (2009). Distance education: Definition and glossary of terms (3rd
ed.). Charlotte, NC: Information Age.
Sherow, S. & Wedemeyer, C. (1990). Origin of distance education in the United States. In
Garrison, D. R. & Shale, D. (eds.), Education at a distance: from issues to practice (p. 7-22).
Melbourne, FL: Krieger.
Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2012). Teaching and learning at a
distance: Foundations of distance education (5th ed.). Boston, MA: Pearson.
Swan, K. & Shih, L. F. (2005). On the nature and development of social presence in online
course discussion. Journal of Asynchronous Learning Networks 9(3).
Wedemeyer, C. (1981). Learning at the backdoor. Madison, WI: University of Wisconsin Press.
Wu, D. & Hiltz, S. R. (2004). Predicting learning form asynchronous online discussions. Journal
of Asynchronous Learning Networks 8(2), pp. 139-152.
Thursday, January 10, 2013
Defining Distance Learning
Prior to this past week, it had never occurred to me to
define distance learning. It was like
flirting; although I never tried to
define it, I had a functional notion of what it was and believed I knew when I
was doing it. Then I started this class, and I realized that although distance
learning (and flirting) is ultimately individually defined, there are in fact distinct
inclusion criteria that must be met in order to qualify as genuine distance learning. I’ll come back to
that in a moment.
First, I want to offer my best definition of distance
learning as I understood it prior to beginning this course.
Distance learning is learning in which:
- The learner may self-select areas of inquiry.
- The learner accesses instructional materials and completes assignments independently.
- The learner may receive feedback from an instructor, facilitator, or electronic medium.
The information I have digested this week has both narrowed and expanded my understanding of
distance learning, and I now realize that some of what I used to consider
distance learning was in fact “self-study at a distance,” (Distance Education:
The Next Generation, 2012), and there is more to distance learning than I
previously thought.
I would now define distance learning as learning in which:
- The learner is separated from the instructor and classmates by physical distance and/or time.
- The learner accesses instructional materials and completes assignments independently or in collaboration with classmates, using applicable media and other technology.
- The learner, instructor, and classmates frequently interact via multimedia, in person, or both.
- The instructor provides feedback to the learner and challenges the learner to link previous knowledge with new learning.
- The instructor encourages the learner to take an active role in the learning process.
- The instructor challenges the learner to demonstrate understanding through analysis, presentation, and application of new learning.
- The instructor provides opportunities for the learner to engage in self-reflection and meta-cognition.
- The learning experience is as equivalent as possible or reasonable to a traditional learning experience.
When I reflect on the evolution of distance education to
date, I can’t help but believe that the future of distance learning is robust,
multi-dimensional, and somewhat inconceivable. Robust in that emerging technologies become more deeply embedded in
our lives almost every day, and as those technologies broaden the human
experience, we come to expect greater breadth and depth, it becomes our new
normal. “In addition to economics and politics, the growth and impact of distance
education is directly linked to the availability of new technologies,” (Simonson, Smaldino, Albright, &
Zvacek, 2012. pp. 17-18.). Multi-dimensional
because as technology continues to bring diverse information to us faster and
easier than ever before, we come to expect more of the same, only better. We expect
to be able to tap into information all the time, and we expect it to come faster,
easier, more customized, and more mobile. And flexible. And interactive. And
engaging. Oh, and “smart.” Inconceivable
because “life must be lived forward, but can only be understood backwards,” (Soren
Kierkegaard, 1813-1836).
I don’t seriously consider the possibility that distance
learning will extinguish in the foreseeable future. Technology has diminished
our perception of the distance between hemispheres and made real the notion of
being able to study virtually any area of inquiry virtually. According to the Sloan Consortium Survey (Allen & Seaman, 2010):
- The majority of chief academic officers believe that the learning outcomes in online courses will equal or exceed that of face-to-face courses within 3 years.
- The overall growth rate for enrollment in online courses is expected to be 20%.
- Given an option, students will enroll in online courses.
How far will distance learning evolve, and what will the
experience be? That, of course, remains to be seen. I believe that all parties involved
(instructors, learners, instructional designers, and other resources) contribute
to the continual evolution of distance learning, specifically the form,
function, delivery, and evaluation of distance learning. “One
challenge to ID is to determine how learners interact with the various
e-learning instructional models and the contexts in which they do so,” (Moller,
Foshay, and Huett, 2008. p. 74), which will play a role in shaping the future of
distance learning.
The Mind Map below is a graphic depiction of my understanding of distance learning. It is designed to show some of the important ways I, as a distance learner, connect with instructors, technology, and classmates, and how we harness variables such as technology and experience to enrich our connections and the learning experience.
References
Allen, I., & Seaman, J. (2010). Learning on Demand: Online education in the United States, 2009. Wellesley, MA: Sloan Consortium.
Laureate Education, Inc. (Producer). (2009). Distance Education: The Next Generation [DVD]. Baltimore, MD: Author
Laureate Education, Inc. (Producer). (2009). Distance Education: The Next Generation [DVD]. Baltimore, MD: Author
Moller, L.,
Foshay, W., & Huett, J. (2008). The evolution of distance education:
Implications for instructional design on the potential of the web (Part 1:
Training and development). TechTrends, 52(3), 70–75.
Simonson, M.,
Smaldino, S., Albright, M., & Zvacek, S. (2012). Teaching and learning
at a distance: Foundations of distance education (5th ed.) Boston, MA:
Pearson.
Subscribe to:
Posts (Atom)
