Tag: assessment

Mini-Scenarios for Assessment

Scenario-based learning often means complex branching or simulations, but it doesn’t always have to be that way. You can use mini-scenarios to make your assessments more relevant and valuable. One of the big advantages of using mini-scenarios is that they’re fast and easy to build. You don’t need any special tools; any tool that can create a multiple choice question can be used for mini-scenarios.

Traditional Assessment or Mini Scenario

Imagine you’re creating a course for managers about motivation and using rewards effectively. You could ask a fairly typical comprehension question like this:

What is the best strategy for encouraging long-term behavior change in your employees?

  1. Threaten punishment for anyone not changing their behavior
  2. Offer a small reward for changing behavior
  3. Offer a large reward for changing behavior

If you have introduced this concept already, this question probably aligns to that content and to your learning objective. However, it’s very abstract. Compare that question to this one:

Andrew is a sales manager who has been struggling to motivate his team to better performance. He sent his team to a conference where they learned about sharing stories about previous happy customers to improve sales. A few salespeople really like using this technique, but he wants everyone to start using it more. In the long term, he wants to change their attitudes about the technique.

What should Andrew do to encourage his team?

  1. Threaten punishment for anyone not using storytelling
  2. Offer a small reward for using storytelling
  3. Offer a large reward for using storytelling

In the second example, a mini-scenario sets up the question. This provides context and makes it a concrete situation with a problem to solve rather than an abstract comprehension question. Now this is about applying the concept in a relevant situation rather than just remembering what you read.

Using a mini-scenario added a total of four sentences to the question. This is actually longer than many of my mini-scenarios; one or two sentences are often enough. You could use this assessment question in any tool, even using the built-in quizzing for many LMSs. It doesn’t take much more time to write than a traditional multiple choice question.

I have often found mini-scenarios useful for helping clients try out scenario-based learning without having to commit to something more complex and expensive. This is a way they can dip their toe in the water without having to do a completely scenario-based course. Even in a fairly traditional linear e-learning course, using mini-scenarios can make your knowledge checks more engaging and effective.

Image Credit: Graphic Stock (unlimited downloads $99/year)

ID and E-Learning Links (8/23/15)

Posted from Diigo. The rest of my favorite links are here.

Intrinsic and Instructional Feedback in Learning Scenarios

A few years ago, I was a judge for a competition on scenario-based learning. While there were a few terrific submissions, I thought many of the courses missed the whole point of scenario-based learning. They started out fine: they provided some sort of realistic context and asked learners to make a decision. Then, instead of showing them the consequences of their decision, they just provided feedback as if it was any other multiple choice assessment. “Correct, that is the best decision.” Blah. Boring. And ineffective.

In her book Scenario-based e-Learning: Evidence-Based Guidelines for Online Workforce Learning, Ruth Clark labels the two types of feedback “intrinsic” and “instructional.” Instructional feedback is what we see all the time in e-learning; it’s feedback that tells you what was right or wrong and possibly guides or coaches you about how to improve.

With intrinsic feedback, the learning environment responds to decisions and action choices in ways that mirror the real world. For example, if a learner responds rudely to a customer, he will see and hear an unhappy customer response. Intrinsic feedback gives the learner an opportunity to try, fail, and experience the results of errors in a safe environment.

Intrinsic feedback is one of the features of scenario-based learning that sets it apart from traditional e-learning. When you show learners the consequences of their actions, they can immediately see why it matters. The principles or process that you’re teaching isn’t just abstract content anymore; it’s something with real world implications and it matters if they get it wrong. It’s more engaging to receive intrinsic feedback. Learners are also more likely to remember the content because they’ve already seen what could happen if they don’t make the right choices.

Intrinsic feedback can take a number of forms. Customer reactions (verbal and nonverbal), patient health outcomes improving, sales figures dropping, a machine starting to work again, and other environmental responses can be intrinsic feedback. The example below contains three pieces of intrinsic feedback, all on the left side: a facial expression, a conversation response, and a motivation meter at the bottom.

Screenshot of a branching scenario with intrinsic and instructional feedbackIn this example, learners are trying to convince someone to make healthier eating choices using motivational interviewing. Motivation level is an “invisible” factor, so I made it visible with a motivation indicator in the lower left corner. As learners make better choices and the patient feels more motivated to change, the motivation meter shows their progress.

Scenarios can also use instructional feedback. In the above example, a coach at the top provides instructional feedback and guidance on learners’ choices. Clark recommends using both intrinsic and instructional feedback in most situations.

One issue with instructional feedback is that it can break the realism of a scenario. Using a coach can help alleviate that problem, as can having learners ask for advice from people inside a scenario (a manager, an HR rep, another worker, etc.). Using a conversational tone for the instructional feedback also helps keep it within the scenario. Instructional feedback in a scenario often doesn’t need to explicitly say that a choice was correct or incorrect; that’s clear enough from the intrinsic feedback. Focus your instructional feedback on explaining why a choice was effective or how it could have been better.

Feedback can also be delayed rather than happening immediately. Clark recommends more immediate feedback for novices but delayed feedback for experts or more advanced learners. Depending on the audience, for some branching scenarios I do immediate intrinsic feedback for each choice learners make. When learners make bad choices that cause them fail and they need to restart the scenario, they receive instructional feedback with guidance on how to improve on their next attempt. They might be able to make two or three bad choices in a row before they hit a dead end in the scenario, so the instructional feedback is delayed. It keeps the momentum of the scenario moving forward but still provides support to learners to help them improve.

If you’re building scenario-based learning, don’t leave out the intrinsic feedback! Your learners will thank you.

CCK08: Connectivism, Equity, and Equality

In Groups Vs Networks: The Class Struggle Continues, Stephen Downes makes this statement about assessment:

I want to change the system of assessment in schools because right now we have tests and things like that that are scrupulously fair, particularly distance learning where we outline the objectives the performance metrics and the outcomes and all of that. I want to scrap that system. I want testing to be done by at random by comments from your peers and other people and strangers based on no criteria whatsoever and applied unequally and unfairly.

I found this a little jarring at first. Don’t we want things to be fair, to apply the same rules to everyone?

But applying the rules uniformly to everyone isn’t fair. The rules of baseball require that people run between the bases. Would you ask someone in a wheelchair to get up and run though, just because the rules say so? No, of course not. It’s absurd, not fair.

Most of the time, our educational system is set up with equality held up as the ideal. Everyone should be treated equally; we should hold everyone to the same standards. No exceptions should be made for individuals to bend the rules. In the US, NCLB is a prime example of this: every child is expected to meet the grade level goals, regardless of learning or other disabilities. We start from the assumption that everyone will learn and be assessed equally.

A better ideal for the system would be equity. We can move the emphasis away from applying the rules consistently across the board to giving people what they need as individuals to be successful. We should recognize that people do have obstacles to overcome and provide support for them to get around those obstacles. Being in a wheelchair means someone won’t run, but it certainly doesn’t mean they can’t participate in any sports.

The ALA article Equality and Equity of Access: What’s the Difference? describes equality as “fairness as uniform distribution” and equity as “fairness as justice.”

It occurred to me as I read Stephen’s ideas about assessment that connectivism may be a better way to get to the ideal of equity. It’s better for equity and accessibility when you don’t start from the assumption that everyone will learn and be assessed in the same way. If we start with the assumption that individuals will find their own path in learning, and that our job is to give them lots of opportunities and ways to participate, we’re more likely to help people get past their obstacles.

The CCK08 class is modeling that approach of letting people find their own path and giving them a chance for equity. Everything Stephen talks about with valuing diversity over uniformity reinforces that idea. The 2000 people can figure out what works best for them–lots of time in the Moodle forums or none, multiple blog posts or just reading and lurking, concept maps or word clouds, live sessions or only asynchronous. It’s what allows me to still be a participant in this class even though I knew I’d be out for a few weeks while I moved. I could take that break when I needed and step back in now.

I don’t know whether anyone in the course is visually or hearing impaired, but I can’t see any reason why they couldn’t find ways to actively participate and learn. Not everything is accessible to everyone, but you don’t need to see every image or hear the audio presentations to find value in the course.

I do wonder though–with the course so open and flexible, and with so many people participating, how much diversity is actually represented by the participants of the CCK08 class?