Tag: branching scenario

Feedback in Branching Scenarios: What Works for Novices, Experts, and Everyone

When we provide feedback in branching scenarios, we have several questions to consider.

  • Should we provide consequences (intrinsic feedback) or coaching (instructional feedback)?
  • Should we provide immediate feedback or delayed feedback?
  • What works for novices versus experts?

Intrinsic and Instructional Feedback

In Scenario-based e-Learning: Evidence-Based Guidelines for Online Workforce Learning, Ruth Clark recommends combining intrinsic and instructional feedback.

Intrinsic feedback is the consequences for an action. It’s what happens because of the learner’s decisions. If you have a scenario where an employee falls off a ladder, a customer agrees to buy a more expensive product, or a patient recovers from a medical emergency, that’s intrinsic feedback. You show the learner what happens.

Instructional feedback is coaching that tells the learner about their choice rather than showing them. In a branching scenario, instructional feedback could come from a coach or character that guides learners. Instructional feedback doesn’t necessarily have to mean telling people directly if their choice was correct or incorrect. Learners should be able to figure that out from the intrinsic feedback. Instead, instructional feedback can focus on addressing misunderstanding or explaining why a choice had a certain result.

Novices may need more instructional feedback than experts. Experts are less likely to have problems with cognitive load from sorting through multiple pieces of information in a scenario. Experts are better at diagnosing their own problems based on contextual information like intrinsic feedback. Novices, on the other hand, may need more direct coaching to make sense of the intrinsic feedback, especially when they fail a scenario.

Immediate and Delayed Feedback

When we build branching scenarios, immediate consequences provide realism and keep learners engaged. Every time learners make a decision, something happens: the customer responds, the equipment breaks, or sales go up.

Note that “immediate” here refers to when the learner receives the feedback, not how quick the results would happen in real life. If a learner makes a choice to ignore recommended equipment maintenance to save money, you could jump ahead in time three months to show that equipment breaking and costing more money in the long run. As long as you show the feedback right away, it’s immediate because it gives learner information about their choice immediately.

Delayed consequences happen in branching scenarios when you show one consequence immediately, but a different consequence appears later.

For example, let’s take a scenario where a manager asks an ID to create training. The learner chooses to have the ID start building it right away, trusting that the team requesting the training knows their needs without further analysis.

  • The immediate consequence is that the ID’s manager is happy.
  • The delayed consequence is that the ID creates ineffective training that doesn’t actually solve the business problem.

You can also use delayed feedback, or coaching delivered to the learner later. In his report on Providing Learners with Feedback, Will Thalheimer suggests that feedback should be provided before learners try again. While that research was more related to retaking tests, I think that’s a good guideline for scenario-based learning. If learners fail a scenario and are asked to try again, give them some feedback to help them learn from their mistakes and make better choices next time.

Novices may benefit from more immediate feedback and coaching, while experts may be fine just receiving coaching at the end of a scenario.

Recommendations for Feedback

Here are my overall recommendations for feedback in scenario-based learning. These are based on a combination of research reviews from Clark and Thalheimer, along with recommendations from Cathy Moore, Michael Allen, and others, plus my own experience.

scenario_feedback

For Everyone

  • Provide frequent, immediate consequences that show learners what happens as a result of their decisions.
  • Provide coaching before learners retry a scenario.
  • Use delayed consequences in scenarios where they are realistic, although note that novices may need more coaching to help them understand delayed consequences.

For Novices

  • Provide immediate coaching for novices, especially to correct misconceptions or incorrect strategy selection.

For Experts

  • Use more delayed coaching with expert learners.

Don’t Assume the Recommendations are Perfect

None of these recommendations are correct 100% of the time for every situation or every group of learners. I’m fairly confident recommending frequent immediate consequences and coaching before a retry, but you may find exceptions even to those recommendations. The research on feedback is sometimes contradictory, so there is little firm guidance.

To quote Will Thalheimer, describing conflicting research results, “First, it tells us that we should be skeptical of absolutism. In particular, it would be perilous for us to say, ‘Immediate feedback is always better,’ or, ‘Delayed feedback is always better.'”

Let’s use the research to guide our decisions in providing feedback, but let’s also acknowledge that the research has limitations. Sometimes we have to use our best judgement on how to best support our learners.

 

Better Feedback for Scenario-Based eLearning Presentation

If you weren’t able to attend my session at the Learning Solutions Conference in Orlando, you can still hear me speaking on this topic. This recording is from a virtual version of the same presentation which I gave to the Online Network of Independent Learning Professionals on March 1 to prepare for the conference.

If you’re reading this in email or RSS and the video doesn’t appear above, try watching it directly on YouTube.

Watch for my next post where I’ll share some of the things I learned at the conference.

Interested in more on this topic? Read all my posts on Storytelling and Scenarios, including several on using feedback to support learning.

Learning Solutions Conference & Expo

Better Feedback for Scenario-Based eLearning Session Trailer

I’m presenting at the eLearning Guild’s Learning Solutions Conference again this year on “Better Feedback for Scenario-Based eLearning.”

You can watch a two-minute trailer for my session (if the video isn’t embedded below, watch it on YouTube).

When you create a scenario, you work hard to make it realistic and relevant for your learners. Unfortunately, even otherwise engaging scenarios sometimes include abstract feedback like “Incorrect. Please try again.” Simply saying the choice is right or wrong can make learners lose interest and focus, and it doesn’t help them learn from their mistakes.

You will learn how to show learners the consequences of their decisions rather than telling them they’re right or wrong in scenario-based eLearning. This is the difference between “intrinsic feedback” and “instructional feedback.” We will explore several different options for intrinsic feedback, such as progress meters, character responses, and environmental changes. You’ll learn guidelines for when to use immediate feedback and when to delay the feedback in scenarios. We’ll discuss how to design feedback to meet the needs of both novice and expert learners. You’ll also learn when direct instructional feedback is beneficial for learning.

In this presentation, you’ll learn:

  • multiple methods to show the consequences of decisions in scenarios
  • when to use immediate or delayed feedback
  • how to provide appropriate feedback for novice and expert learners
  • when to use intrinsic feedback (showing consequences) and instructional feedback (direct coaching)
  • how to work with SMEs to get information to provide realistic consequences
  • how to write better feedback for short scenarios and complex branching scenarios

Read More about Feedback

This session draws from several previous blog posts (as well as some additional information from other sources).

If you’re attending the Learning Solutions Conference, I hope to see you there!

Using Time as Scenario Feedback

Nicole is creating a branching scenario practicing communication techniques for nutrition counselors to better understand their clients’ goals. She has written a simulated conversation between a counselor and a client. Her SME, Brian, provided this feedback after reviewing the prototype.

The conversation overall does a good job giving plausible choices for questions and showing realistic responses from the client. I want this to be really exciting for learners, like a game. Let’s add a timer for each decision. That way, they’ll be motivated to answer quickly and keep pushing through the scenario.

What do you think? How should Nicole respond to Brian?

A. That’s a great idea! I think that will enhance the learner experience.

B. I’m not sure. Let me do some research.

C. Timing might not be the best form of feedback for this particular course.

Remember your answer; we’ll come back to this question at the end of the post.

Using Time as Scenario Feedback

When Time is Effective

Time can be a very effective consequence in some learning situations. Check out the Lifesaver training on what to do in emergency situations for an example with effective use of time as feedback.

In the first scenario with Jake, you help someone in cardiac arrest. Each question has a 5 second timer, and you are scored for both accuracy and speed.

6/6 Right First Time - Avg Speed 1.32s

Later in the scenario, you simulate performing CPR by pressing two keys on your keyboard in the same rhythm as CPR. While you practice, you see a scale from good to bad showing how close you are to the ideal timing. This lets you adjust your rhythm. After you finish the 30 seconds of simulated CPR, you see a percentage score for your accuracy.

Scale showing good and bad for speed

This feedback works in the Lifesaver training because timing really is a critical part of the skill being taught. Speed of response matters in these emergency situations, as does knowing the right rhythm for CPR.

Time can work for other skills too, like manufacturing, making sandwiches in a chain restaurant, or safety training.

When Time is Counterproductive

If the skill you’re practicing and assessing requires critical thinking and careful consideration, measuring time can be counterproductive. For simulated conversations where you want learners to pause and think about their options, it’s better to not use a timer.

You might be thinking, “But in a real conversation, people need to think quickly. Doesn’t that mean we should use timers?” That’s a question about fluency, which requires more practice over time. If your goal is to get people to that point of fluency, you might add a timer, but not for the initial practice. Teach the skill without a timer first, then provide additional practice opportunities to build fluency and speed in the skill.

Do Timers Improve Motivation?

Does a timer increase motivation for getting the right answer? Maybe, if the learners are already motivated prior to starting and time makes sense in the context of the activity. Many games use time as a way to keep players engaged and excited.

I suspect in practice that unnecessary timers encourage people to guess randomly for whatever they can click fastest. Time may actually decrease learners’ motivation to be truly cognitively engaged with the learning experience. They may be more motivated just to click and get through it quickly than to read carefully and understand their decisions.

Accessibility

Timers can create challenges for accessibility. Learners with visually impairments who use a screen reader and keyboard navigation will generally need more time to answer. Learners with  mobility impairment may have trouble manipulating a mouse or keyboard quickly. Depending on your audience, adding timers may prevent some learners from being successful in your elearning courses, even if they could do the real task (like having a conversation) without problem.

Revisiting the Communication Scenario Example

Think back to Nicole’s scenario at the beginning of this post. She’s teaching communication skills to nutrition counselors, using a simulated conversation. Her SME, Brian, suggested adding a timer.

What do you think? Does a timer seem helpful in this situation?

Probably not. In this training, it’s more important for learners to think carefully about their choices and responses than to be speedy. Feedback like the expression of the client or a scale showing the client’s motivation to change their eating behavior would be more beneficial than feedback on how quick they are.

Your Examples?

Time can work as feedback in learning scenarios, but it should be used sparingly, and only when it is actually relevant to the skill being practiced or assessed.

Do you have any examples of time used successfully as feedback in a scenario? I’d love to see some more samples. Share them in the comments.

 

Branching Scenario Prototype in Twine

I built this branching scenario in the open source tool Twine. This scenario is moderately complex, with a total of 17 pages (or passages in Twine terminology) and 8 different endings. The ideal path has 5 decisions to reach the best conclusion.

I generally use Twine as a prototype for review and testing purposes. You can use Twine as the finished product though, especially if you do some formatting to make it look better. This is currently pretty rough (just text on a white background), but that’s OK for a prototype.

If you use Twine as a prototyping tool, you can build the finished version in Captivate, Storyline, or another tool of your choice.

Try the scenario out yourself by clicking below (the scenario will open in a new tab).

Click to open the scenario in a new tab.

This is the map of the entire scenario. You can see how many of the choices are reused.

Twine map of the entire scenario

Want to learn how I created this?

Read the previous posts in the series to see my process for creating this scenario.

 

Can’t get enough? Check out all of my posts on Storytelling and Scenarios.