Intrinsic and Instructional Feedback in Learning Scenarios

A few years ago, I was a judge for a competition on scenario-based learning. While there were a few terrific submissions, I thought many of the courses missed the whole point of scenario-based learning. They started out fine: they provided some sort of realistic context and asked learners to make a decision. Then, instead of showing them the consequences of their decision, they just provided feedback as if it was any other multiple choice assessment. “Correct, that is the best decision.” Blah. Boring. And ineffective.

In her book Scenario-based e-Learning: Evidence-Based Guidelines for Online Workforce Learning, Ruth Clark labels the two types of feedback “intrinsic” and “instructional.” Instructional feedback is what we see all the time in e-learning; it’s feedback that tells you what was right or wrong and possibly guides or coaches you about how to improve.

With intrinsic feedback, the learning environment responds to decisions and action choices in ways that mirror the real world. For example, if a learner responds rudely to a customer, he will see and hear an unhappy customer response. Intrinsic feedback gives the learner an opportunity to try, fail, and experience the results of errors in a safe environment.

Intrinsic feedback is one of the features of scenario-based learning that sets it apart from traditional e-learning. When you show learners the consequences of their actions, they can immediately see why it matters. The principles or process that you’re teaching isn’t just abstract content anymore; it’s something with real world implications and it matters if they get it wrong. It’s more engaging to receive intrinsic feedback. Learners are also more likely to remember the content because they’ve already seen what could happen if they don’t make the right choices.

Intrinsic feedback can take a number of forms. Customer reactions (verbal and nonverbal), patient health outcomes improving, sales figures dropping, a machine starting to work again, and other environmental responses can be intrinsic feedback. The example below contains three pieces of intrinsic feedback, all on the left side: a facial expression, a conversation response, and a motivation meter at the bottom.

Screenshot of a branching scenario with intrinsic and instructional feedbackIn this example, learners are trying to convince someone to make healthier eating choices using motivational interviewing. Motivation level is an “invisible” factor, so I made it visible with a motivation indicator in the lower left corner. As learners make better choices and the patient feels more motivated to change, the motivation meter shows their progress.

Scenarios can also use instructional feedback. In the above example, a coach at the top provides instructional feedback and guidance on learners’ choices. Clark recommends using both intrinsic and instructional feedback in most situations.

One issue with instructional feedback is that it can break the realism of a scenario. Using a coach can help alleviate that problem, as can having learners ask for advice from people inside a scenario (a manager, an HR rep, another worker, etc.). Using a conversational tone for the instructional feedback also helps keep it within the scenario. Instructional feedback in a scenario often doesn’t need to explicitly say that a choice was correct or incorrect; that’s clear enough from the intrinsic feedback. Focus your instructional feedback on explaining why a choice was effective or how it could have been better.

Feedback can also be delayed rather than happening immediately. Clark recommends more immediate feedback for novices but delayed feedback for experts or more advanced learners. Depending on the audience, for some branching scenarios I do immediate intrinsic feedback for each choice learners make. When learners make bad choices that cause them fail and they need to restart the scenario, they receive instructional feedback with guidance on how to improve on their next attempt. They might be able to make two or three bad choices in a row before they hit a dead end in the scenario, so the instructional feedback is delayed. It keeps the momentum of the scenario moving forward but still provides support to learners to help them improve.

If you’re building scenario-based learning, don’t leave out the intrinsic feedback! Your learners will thank you.

12+ Books for Instructional Designers

If you’re looking for some reading to improve your skills or get started in the field of instructional design, check out these books.

ID books

General Instructional Design and E-Learning

Design For How People Learn by Julie Dirksen is one of my favorite books in the field. I’ve recommended it many times. It’s easy to read and understand. It makes research about learning accessible in ways you can apply immediately. The illustrations are charming and reinforce the concepts well. Read my review for more details.

The Accidental Instructional Designer by Cammy Bean is especially good for career changers and those who landed in instructional design from other fields. It provides a model for the range of skills that fall under the umbrella of “instructional design.” It includes practical tips on topics such as working with SMEs and avoiding “clicky clicky bling bling” or flashy interactivity and multimedia for the sake of being flashy. The design models in chapter 4 are probably familiar to many with experience in the field but very helpful to beginners who want to do more than just the same type of course and interaction for every situation.

Designing Successful e-Learning by Michael Allen tells you to “Forget What You Know About Instructional Design and Do Something Interesting.” All of Allen’s books are focused on helping people design e-learning that is interactive, engaging, and useful.

e-Learning and the Science of Instruction by Ruth Clark and Richard Mayer is one of the first books on e-learning I bought, and I still refer to it when I need evidence to justify decisions to clients. If you’ve ever wondered if formal or conversational style is better for learning (conversational) or if your on-screen text should replicate what’s on the screen (no, it shouldn’t), this book explains it with the research to back it up. It’s not perfect; the authors do sometimes disregard research that contradicts their own findings, and they sometimes make their principles seem more absolute than they probably are in real life. However, it’s still a solid reference.

First Principles of Instruction: Identifying and Designing Effective, Efficient and Engaging Instructionis David Merrill’s effort to distill the common principles from multiple instructional design theories. A shorter, earlier explanation of these principles is available as a free PDF.

Games and Scenario-Based Learning

The Gamification of Learning and Instruction by Karl Kapp explains how to do more with gamification than just badges and points. Karl summarizes research and game theory and explains how substantive elements of games like narrative can be used to improve learning design. I wrote more about this gamification research previously.

Scenario-based e-Learning by Ruth Clark is similar to eLearning and the Science of Instruction in that it summarizes research findings. This book is specifically focused on developing scenario-based e-learning, including everything from simple branching scenarios to complex simulations.

Learning Communities

Building Online Learning Communities by Rena Palloff and Keith Pratt is aimed more at online instructors than instructional designers, but it’s a wonderful resource for IDs working in higher education or supporting online and blended learning communities.

Digital Habitats: Stewarding Technology for Communities by Etienne Wenger, Nancy White, and John Smith is about how technology can enable communities of practice.

Other Topics

Learning Everywhere by Chad Udell is a fantastic resource on mobile learning, providing everything from a big picture view of broad categories of mobile learning to specific technical considerations and pitfalls. You can read my review of the book for more details.

Show Your Work by Jane Bozarth is full of visuals and explains how to “show your work” by sharing what you’re doing and learning using social tools. The book explains the benefits of creating a culture where people share their processes and discoveries.

E-Learning Uncovered: Adobe Captivate 8 by Diane Elkins, Desiree Pinder, and Tim Slade sat on my desk for multiple weeks because I used it so often that it wasn’t worth bothering to put it back on the shelf. This book was an immense help to me in learning Captivate 8. I’m sure their other books on Storyline 2, Lectora, etc. are equally valuable.

More Reading Lists

If that’s not enough, these reading lists will give you additional ideas.

Your Selections

Did I miss one of your favorite books? Leave a comment with your suggestions.

No, I Won’t “Tweak” Your PowerPoint Slides

I got a call from a prospective client. She and I spoke briefly once when she was looking for a pool of instructional designers to call on for specific projects, but we haven’t worked together yet. In fact, I’m pretty sure she’s never worked with any instructional designers before.

“Hi, Christy, it’s Lynn. I need some help finishing up an e-learning course. The PowerPoint slides just need to be tweaked—editing the onscreen text, adding some animation, prepping the script for voice over recording, and syncing everything together. Are you available?”

“Your timing is good, Lynn. I’m just wrapping up some other projects and have some time now. Tell me some more about this project.”

“It’s about 200 slides. Here, let me email them to you now so you can take a look. The content is basically finished, and it just needs some polish. Can you do that?”

“OK, I’m looking over the slides now.” Some parts of the presentation have good visuals, but large sections are text obviously copied word-for-word from the employee handbook. Often, the same text is in the voice over notes as on screen. The slides contain no practice activities or assessments. “Is this a face-to-face course that you want to convert to online?” I ask.

Dense text on a slide describing benefits. No amount of "tweaking" will make this much text effective e-learning.

No amount of “tweaking” will make this effective e-learning.

She responds, “Yes, this used to be taught in a classroom. It’s part of the new employee orientation. We have a lot of new employees coming in, and we don’t always have a trainer available. Frankly, some of the trainers are better than others. We want to make sure everyone has the same experience. I had the best trainer write the script out in the slide notes for the narration so it would be just like what he teaches in class.”

“Well, I could do proofreading and animation to just tweak the slides, but I’m not sure that would be really effective. I don’t know what your budget is, but I think this course could benefit from some actual analysis and instructional design.”

“What do you mean by ‘actual analysis and instructional design’?”

“As an instructional designer, I don’t usually do projects where I’m just brought on at the end to tweak slides. I’m typically brought on board shortly after a client decides, ‘We need a course!’ Starting right from the beginning, I work with you to analyze the need, design the instruction, develop the multimedia, and manage the project until launch. I do a kick-off call with you to find out your needs. We talk about what business problem we’re trying to solve; needing a course isn’t a true business problem, so I work to uncover WHY you decided you need a course. For example, for an orientation like this, I’d want to know what’s working and not working in your current orientation. What do people leave orientation and still have problems with? What questions keep coming up over and over to HR?”

Lynn replies, “There are tons of questions with the benefits plan. People just don’t understand it, even after they read the handbook. HR ends up spending a lot of time walking people through all the options.”

“If people don’t understand it right now from reading the handbook, do you think they’ll understand it any better by having someone read it to them?”

“Hmm. I guess not.”

“Right now, you would probably get about the same results from having your new employees read the handbook on their own and take a quiz afterward. The slides are basically a pretty version of your handbook. Reading the handbook and taking a quiz wouldn’t only be cheaper to develop; it would be faster for employees to complete. People read faster than they can listen to voice over, so they can consume the same amount of content in less time by reading the document rather than watching and listening to the same thing online.”

“I see your point. But we don’t really have the budget to do a course from scratch.”

“You know, you probably will spend more upfront to do analysis and develop a more effective course. However, the final orientation would probably be half its current length because it would be focused on what employees need on day one to get started. That means employees would spend less time in orientation and be ready to do real work faster. We could also focus on the problems you really need to solve. If we can reduce the number of questions and problems HR has to deal with, we can free them up to do other work. That can save your company money in the long run even though the initial costs are higher.”

“You know, I might be able to justify that. Cutting down the time for new employees to get up and running is a big deal right now, so if we can help with that, I might be able to find some more budget. I need to talk to HR some more to find out if they have any other issues with new employees. I really thought instructional designers just did a little multimedia work at the end of the process. I didn’t realize you did so much.”

“Instructional design is more about being a partner to help you solve problems than just making slides pretty. You asked me to be a handyman and touch up some peeling paint, but I’m really an architect who can design you a house that better meets your needs.”

“Let me work on the budget and get back to you in a few days so we can talk about scope, OK? I’m not sure I can get enough to do everything you’d usually design, but maybe we can get something to do more analysis.”

“Sound good. I’ll look forward to hearing from you soon.”

This conversation is fictionalized, but it’s based on several real experiences. How do you handle it when someone asks you to just “tweak the slides”? How do you shift the conversation from just being an order taker to doing real instructional design work?

My History of Live Blogged Notes

When I attend webinars or participate in online courses and conferences, I usually live blog my notes. That helps me remember what I attended and what I learned, and it lets me share that knowledge with others. In a recent discussion about how I have learned about instructional design without getting a master’s degree, someone asked me what courses and webinars I’ve attended. Because I have done so much live blogging, I was able to provide proof of my ongoing professional development efforts. These posts go back to 2007, so some of the content and references are dated. Generally newer posts are at the top of each category.

Woman-typing-on-laptop-cropped

Storytelling and Scenario-Based Learning

Synchronous Learning

 Attention and Motivation

Trends and Future Predictions

Games and Simulations

LMSs and Other Tools

Learning Communities

Other Topics

Image credit: Matthew Bowden http://www.digitallyrefreshing.com (http://www.sxc.hu/photo/145972) [Attribution], via Wikimedia Commons

Voice Over Script Review Checklist

I’ve written several posts with tips on how to write voice over scripts. This review checklist summarizes all of the tips from the previous three posts into a single Word document you can download and use yourself.

Voice Over Checklist

Voice Over Script Review Checklist

Feel free to edit this document to match the requirements of your specific organization as long as you retain attribution to me with a link. If you improve this document, I’d love to hear about it.

Here is the complete list from the checklist:

  • The script has been read aloud
  • Script flows well; no awkward or clunky sentences
  • No grammar errors
  • Conversational tone
  • First and/or second person (I, we, you) are used
  • Contractions are used if style allows
  • No overly complicated sentences; variety of sentence length with shorter sentences
  • Pronunciation guides included for jargon, abbreviations, acronyms, and numbers
  • Emphasis in sentences marked as needed
  • Punctuate to mark pauses
  • Readable spacing, font, and font size (at least 12-14 pts)
  • Screen names clearly labeled
  • Numbers are written out as you want them said
  • Lists are written in conversational sentences (first choice) or punctuated for clarity and ease of reading aloud (second choice)
  • Serial comma used for all lists
  • Latin abbreviations are written out or noted: e.g. (“for example”), i.e. (“that is”), and etc. (“et cetera” or “and so on”)

For further explanation of the above points, review the previous posts in the series:

  1. Writing Style Tips for Voice Over Scripts
  2. Formatting Tips for Voice Over Scripts
  3. Voice Over Script Pitfalls

I want to give Jill Goldman of Goldivox one more shout out for being so helpful in putting these posts together.