Debunker Club Works to Dispel the Corrupted Cone of Learning

A new group called The Debunker Club is working to dispel myths and misinformation in the learning field. From their website:

The Debunker Club is an experiment in professional responsibility. Anyone who’s interested may join as long as they agree to the following:

  1. I would like to see less misinformation in the learning field.
  2. I will invest some of my time in learning and seeking the truth, from sources like peer-reviewed scientific research or translations of that research.
  3. I will politely, but actively, provide feedback to those who transmit misinformation.
  4. At least once a year, I will seek out providers of misinformation and provide them with polite feedback, asking them to stop transmitting their misinformation.
  5. I will be open to counter feedback, listening to understand opposing viewpoints. I will provide counter-evidence and argument when warranted.

This year, coinciding with April Fool’s Day 2015, the Debunker Club is running an experiment. We’re making a concerted effort to contact people who have shared the Cone of Experience (also known as the Cone of Learning or the Pyramid of Learning).

Many iterations of this cone exist. A Google image search for “cone of learning” returns dozens of results, most of which are false. If you’ve seen something like this that said, “People remember 10% of what they read, 20% of what they hear, 30% of what they see” etc., you’ve seen a variation on this theme.

Image search results for Cone of Learning

The original cone was developed by Edgar Dale and didn’t include any numbers. The later versions are the “corrupted cone” with fictitious statistics added.  Will Thalheimer’s post from 2009 debunking these claims is where I learned it was incorrect. Common sense might give you a hint that these numbers aren’t really based in research though. Think about it–how many times have you see research where all the categories broke into even 10% segments?

As part of the Debunker Club’s efforts, I discovered a post on Dane’s Education Blog called A Hierarchy of Learning. Although this post cites a great debunking article (Tales of the Undead…Learning Theories: The Learning Pyramid), the blog author only says that he “appreciate what it conveys.”

I left the following comment on his Learning Pyramid post.

Thanks for collecting so many resources on your blog. I can see that you’ve worked really hard to share many links and ideas with your readers.

However, the information above, though it may appear to have scientific support, has been exhaustively researched and found to have no basis in science. In fact, the “Tales of the Undead” link you cite debunks it.

An article from the scientific journal Educational Technology shows no research backing for the information. (Subramony, D., Molenda, M., Betrus, A., and Thalheimer, W. (2014). The Mythical Retention Chart and the Corruption of Dale’s Cone of Experience. Educational Technology, Nov/Dec 2014, 54(6), 6-16.)

The information presented is likely to produce more harm than good, promoting poor learning designs and hurting learners.

While we might abstract some beneficial notions from the percentages portrayed in the misleading information — namely that encouraging realistic practice has benefits — there are numerous faulty concepts within the bogus percentages that can do real harm. For example, by having people think that there are benefits to seeing over hearing, or hearing over reading, we are sending completely wrong messages about how learning works.

Most importantly, recent advances in learning science have really come together over the last two decades. The misleading information was first reported in 1914, with no research backing. It’s better to follow more recent findings than information that has no scientific basis. See, for example, the book Make it Stick by Brown, Roediger, and McDaniel. Julie Dirksen’s Design for How People Learn is another great selection.

I’m part of a larger community of folks called the Debunker Club who are attempting to encourage the use of proven, scientifically-based learning factors in the learning field.

I’m going to be posting about this misleading information on my blog. I hope you’ll comment and respond to my post if you wish. I (and the debunker community in general) want to learn how other people feel about the issues and ideas surrounding the original information and our approach to debunking myths and sharing evidence.

If you’re interested in dispelling misinformation and improving the learning field, please join the Debunker Club and participate in the conversation.

Intrinsic and Instructional Feedback in Learning Scenarios

A few years ago, I was a judge for a competition on scenario-based learning. While there were a few terrific submissions, I thought many of the courses missed the whole point of scenario-based learning. They started out fine: they provided some sort of realistic context and asked learners to make a decision. Then, instead of showing them the consequences of their decision, they just provided feedback as if it was any other multiple choice assessment. “Correct, that is the best decision.” Blah. Boring. And ineffective.

In her book Scenario-based e-Learning: Evidence-Based Guidelines for Online Workforce Learning, Ruth Clark labels the two types of feedback “intrinsic” and “instructional.” Instructional feedback is what we see all the time in e-learning; it’s feedback that tells you what was right or wrong and possibly guides or coaches you about how to improve.

With intrinsic feedback, the learning environment responds to decisions and action choices in ways that mirror the real world. For example, if a learner responds rudely to a customer, he will see and hear an unhappy customer response. Intrinsic feedback gives the learner an opportunity to try, fail, and experience the results of errors in a safe environment.

Intrinsic feedback is one of the features of scenario-based learning that sets it apart from traditional e-learning. When you show learners the consequences of their actions, they can immediately see why it matters. The principles or process that you’re teaching isn’t just abstract content anymore; it’s something with real world implications and it matters if they get it wrong. It’s more engaging to receive intrinsic feedback. Learners are also more likely to remember the content because they’ve already seen what could happen if they don’t make the right choices.

Intrinsic feedback can take a number of forms. Customer reactions (verbal and nonverbal), patient health outcomes improving, sales figures dropping, a machine starting to work again, and other environmental responses can be intrinsic feedback. The example below contains three pieces of intrinsic feedback, all on the left side: a facial expression, a conversation response, and a motivation meter at the bottom.

Screenshot of a branching scenario with intrinsic and instructional feedbackIn this example, learners are trying to convince someone to make healthier eating choices using motivational interviewing. Motivation level is an “invisible” factor, so I made it visible with a motivation indicator in the lower left corner. As learners make better choices and the patient feels more motivated to change, the motivation meter shows their progress.

Scenarios can also use instructional feedback. In the above example, a coach at the top provides instructional feedback and guidance on learners’ choices. Clark recommends using both intrinsic and instructional feedback in most situations.

One issue with instructional feedback is that it can break the realism of a scenario. Using a coach can help alleviate that problem, as can having learners ask for advice from people inside a scenario (a manager, an HR rep, another worker, etc.). Using a conversational tone for the instructional feedback also helps keep it within the scenario. Instructional feedback in a scenario often doesn’t need to explicitly say that a choice was correct or incorrect; that’s clear enough from the intrinsic feedback. Focus your instructional feedback on explaining why a choice was effective or how it could have been better.

Feedback can also be delayed rather than happening immediately. Clark recommends more immediate feedback for novices but delayed feedback for experts or more advanced learners. Depending on the audience, for some branching scenarios I do immediate intrinsic feedback for each choice learners make. When learners make bad choices that cause them fail and they need to restart the scenario, they receive instructional feedback with guidance on how to improve on their next attempt. They might be able to make two or three bad choices in a row before they hit a dead end in the scenario, so the instructional feedback is delayed. It keeps the momentum of the scenario moving forward but still provides support to learners to help them improve.

If you’re building scenario-based learning, don’t leave out the intrinsic feedback! Your learners will thank you.

12+ Books for Instructional Designers

If you’re looking for some reading to improve your skills or get started in the field of instructional design, check out these books.

ID books

General Instructional Design and E-Learning

Design For How People Learn by Julie Dirksen is one of my favorite books in the field. I’ve recommended it many times. It’s easy to read and understand. It makes research about learning accessible in ways you can apply immediately. The illustrations are charming and reinforce the concepts well. Read my review for more details.

The Accidental Instructional Designer by Cammy Bean is especially good for career changers and those who landed in instructional design from other fields. It provides a model for the range of skills that fall under the umbrella of “instructional design.” It includes practical tips on topics such as working with SMEs and avoiding “clicky clicky bling bling” or flashy interactivity and multimedia for the sake of being flashy. The design models in chapter 4 are probably familiar to many with experience in the field but very helpful to beginners who want to do more than just the same type of course and interaction for every situation.

Designing Successful e-Learning by Michael Allen tells you to “Forget What You Know About Instructional Design and Do Something Interesting.” All of Allen’s books are focused on helping people design e-learning that is interactive, engaging, and useful.

e-Learning and the Science of Instruction by Ruth Clark and Richard Mayer is one of the first books on e-learning I bought, and I still refer to it when I need evidence to justify decisions to clients. If you’ve ever wondered if formal or conversational style is better for learning (conversational) or if your on-screen text should replicate what’s on the screen (no, it shouldn’t), this book explains it with the research to back it up. It’s not perfect; the authors do sometimes disregard research that contradicts their own findings, and they sometimes make their principles seem more absolute than they probably are in real life. However, it’s still a solid reference.

First Principles of Instruction: Identifying and Designing Effective, Efficient and Engaging Instructionis David Merrill’s effort to distill the common principles from multiple instructional design theories. A shorter, earlier explanation of these principles is available as a free PDF.

Games and Scenario-Based Learning

The Gamification of Learning and Instruction by Karl Kapp explains how to do more with gamification than just badges and points. Karl summarizes research and game theory and explains how substantive elements of games like narrative can be used to improve learning design. I wrote more about this gamification research previously.

Scenario-based e-Learning by Ruth Clark is similar to eLearning and the Science of Instruction in that it summarizes research findings. This book is specifically focused on developing scenario-based e-learning, including everything from simple branching scenarios to complex simulations.

Learning Communities

Building Online Learning Communities by Rena Palloff and Keith Pratt is aimed more at online instructors than instructional designers, but it’s a wonderful resource for IDs working in higher education or supporting online and blended learning communities.

Digital Habitats: Stewarding Technology for Communities by Etienne Wenger, Nancy White, and John Smith is about how technology can enable communities of practice.

Other Topics

Learning Everywhere by Chad Udell is a fantastic resource on mobile learning, providing everything from a big picture view of broad categories of mobile learning to specific technical considerations and pitfalls. You can read my review of the book for more details.

Show Your Work by Jane Bozarth is full of visuals and explains how to “show your work” by sharing what you’re doing and learning using social tools. The book explains the benefits of creating a culture where people share their processes and discoveries.

E-Learning Uncovered: Adobe Captivate 8 by Diane Elkins, Desiree Pinder, and Tim Slade sat on my desk for multiple weeks because I used it so often that it wasn’t worth bothering to put it back on the shelf. This book was an immense help to me in learning Captivate 8. I’m sure their other books on Storyline 2, Lectora, etc. are equally valuable.

More Reading Lists

If that’s not enough, these reading lists will give you additional ideas.

Your Selections

Did I miss one of your favorite books? Leave a comment with your suggestions.

No, I Won’t “Tweak” Your PowerPoint Slides

I got a call from a prospective client. She and I spoke briefly once when she was looking for a pool of instructional designers to call on for specific projects, but we haven’t worked together yet. In fact, I’m pretty sure she’s never worked with any instructional designers before.

“Hi, Christy, it’s Lynn. I need some help finishing up an e-learning course. The PowerPoint slides just need to be tweaked—editing the onscreen text, adding some animation, prepping the script for voice over recording, and syncing everything together. Are you available?”

“Your timing is good, Lynn. I’m just wrapping up some other projects and have some time now. Tell me some more about this project.”

“It’s about 200 slides. Here, let me email them to you now so you can take a look. The content is basically finished, and it just needs some polish. Can you do that?”

“OK, I’m looking over the slides now.” Some parts of the presentation have good visuals, but large sections are text obviously copied word-for-word from the employee handbook. Often, the same text is in the voice over notes as on screen. The slides contain no practice activities or assessments. “Is this a face-to-face course that you want to convert to online?” I ask.

Dense text on a slide describing benefits. No amount of "tweaking" will make this much text effective e-learning.

No amount of “tweaking” will make this effective e-learning.

She responds, “Yes, this used to be taught in a classroom. It’s part of the new employee orientation. We have a lot of new employees coming in, and we don’t always have a trainer available. Frankly, some of the trainers are better than others. We want to make sure everyone has the same experience. I had the best trainer write the script out in the slide notes for the narration so it would be just like what he teaches in class.”

“Well, I could do proofreading and animation to just tweak the slides, but I’m not sure that would be really effective. I don’t know what your budget is, but I think this course could benefit from some actual analysis and instructional design.”

“What do you mean by ‘actual analysis and instructional design’?”

“As an instructional designer, I don’t usually do projects where I’m just brought on at the end to tweak slides. I’m typically brought on board shortly after a client decides, ‘We need a course!’ Starting right from the beginning, I work with you to analyze the need, design the instruction, develop the multimedia, and manage the project until launch. I do a kick-off call with you to find out your needs. We talk about what business problem we’re trying to solve; needing a course isn’t a true business problem, so I work to uncover WHY you decided you need a course. For example, for an orientation like this, I’d want to know what’s working and not working in your current orientation. What do people leave orientation and still have problems with? What questions keep coming up over and over to HR?”

Lynn replies, “There are tons of questions with the benefits plan. People just don’t understand it, even after they read the handbook. HR ends up spending a lot of time walking people through all the options.”

“If people don’t understand it right now from reading the handbook, do you think they’ll understand it any better by having someone read it to them?”

“Hmm. I guess not.”

“Right now, you would probably get about the same results from having your new employees read the handbook on their own and take a quiz afterward. The slides are basically a pretty version of your handbook. Reading the handbook and taking a quiz wouldn’t only be cheaper to develop; it would be faster for employees to complete. People read faster than they can listen to voice over, so they can consume the same amount of content in less time by reading the document rather than watching and listening to the same thing online.”

“I see your point. But we don’t really have the budget to do a course from scratch.”

“You know, you probably will spend more upfront to do analysis and develop a more effective course. However, the final orientation would probably be half its current length because it would be focused on what employees need on day one to get started. That means employees would spend less time in orientation and be ready to do real work faster. We could also focus on the problems you really need to solve. If we can reduce the number of questions and problems HR has to deal with, we can free them up to do other work. That can save your company money in the long run even though the initial costs are higher.”

“You know, I might be able to justify that. Cutting down the time for new employees to get up and running is a big deal right now, so if we can help with that, I might be able to find some more budget. I need to talk to HR some more to find out if they have any other issues with new employees. I really thought instructional designers just did a little multimedia work at the end of the process. I didn’t realize you did so much.”

“Instructional design is more about being a partner to help you solve problems than just making slides pretty. You asked me to be a handyman and touch up some peeling paint, but I’m really an architect who can design you a house that better meets your needs.”

“Let me work on the budget and get back to you in a few days so we can talk about scope, OK? I’m not sure I can get enough to do everything you’d usually design, but maybe we can get something to do more analysis.”

“Sound good. I’ll look forward to hearing from you soon.”

This conversation is fictionalized, but it’s based on several real experiences. How do you handle it when someone asks you to just “tweak the slides”? How do you shift the conversation from just being an order taker to doing real instructional design work?

My History of Live Blogged Notes

When I attend webinars or participate in online courses and conferences, I usually live blog my notes. That helps me remember what I attended and what I learned, and it lets me share that knowledge with others. In a recent discussion about how I have learned about instructional design without getting a master’s degree, someone asked me what courses and webinars I’ve attended. Because I have done so much live blogging, I was able to provide proof of my ongoing professional development efforts. These posts go back to 2007, so some of the content and references are dated. Generally newer posts are at the top of each category.

Woman-typing-on-laptop-cropped

Storytelling and Scenario-Based Learning

Synchronous Learning

 Attention and Motivation

Trends and Future Predictions

Games and Simulations

LMSs and Other Tools

Learning Communities

Other Topics

Image credit: Matthew Bowden http://www.digitallyrefreshing.com (http://www.sxc.hu/photo/145972) [Attribution], via Wikimedia Commons