Weekly Bookmarks (8/21/11)

• The Human Factor: How Gender Differences Matter in Software Training by Mary Arnold : Learning Solutions Magazine

  If your software training includes time to explore or “tinker,” men and women will have different rates of success. A strategic approach may be better than going through individual features. This research focused on adding new features with an audience who was already familiar with the software; I’m not sure the same training technique would work with beginners with an application.

  tags: e-learning softwaretraining research diversity

  • Tinkering with the spreadsheets seems to be a reasonable approach to working with a new problem, in line with generating and testing alternative strategies to find a solution. In other words, learning.  Women who tinkered with the spreadsheets seemed to be doing just that, and, for them, tinkering predicted more effective problem solving.  Counter-intuitively, though, when men tinkered with the spreadsheet, they were less effective in correcting the errors.  The opposite results seem attributable to the fact that women paused before trying something else, long enough to process the information.
  • In the final experiment, researchers provided a different kind of tutorial — one that emphasized a strategic, rather than a feature-by-feature approach to the problem.
  • Women who participated in this condition were almost as likely to use the new features as the men in the same study, and were able to solve more problems more quickly than women who didn’t use the new features.  Men in this condition were not significantly helped or hindered, which means that it’s possible to prevent a bias against women without introducing a bias against men.

Posted from Diigo. The rest of my favorite links are here.

Weekly Bookmarks (8/7/2011)

• Intel Education: Designing Effective Projects: Thinking Frameworks

  Review of Bloom’s Taxonomy, including problems and the revised version, with information about the differences between factual, conceptual, procedural, and metacognitive knowledge.

  tags: bloom learning education

  • Those teachers who keep a list of question prompts relating to the various levels of Bloom’s Taxonomy undoubtedly do a better job of encouraging higher-order thinking in their students than those who have no such tool. On the other hand, as anyone who has worked with a group of educators to classify a group of questions and learning activities according to the Taxonomy can attest, there is little consensus about what seemingly self-evident terms like “analysis,” or “evaluation” mean. In addition, so many worthwhile activities, such as authentic problems and projects, cannot be mapped to the Taxonomy, and trying to do that would diminish their potential as learning opportunities.

Posted from Diigo. The rest of my favorite links are here.

Questioning Gagné and Bloom’s Relevance

Several weeks ago, I had the pleasure of meeting Lauren Hirsh. Lauren needed to do an informational interview for her masters program, and I needed some new profile pictures. (The pictures turned out terrific; I’m sure I got the better end of the bargain.)

During the interview, Lauren asked some very thoughtful questions about the relationship between theory and practice.

I made this comment as part of the interview:

It’s easy to get caught up in theories without really looking at whether the research support is there. Gagne’s Nine Events of Instruction might be helpful as a designer, but they aren’t really supported; you can skip everything but practice with feedback without much change in results. Learning styles (like visual, auditory, kinesthetic) have much less effect on learning results than other factors, but we often focus on them heavily. Bloom didn’t have any research for his taxonomy, but I still find it useful for my own planning; I just don’t pretend there’s a research-based argument for classifying a verb as application instead of analysis.

As a follow-up question, she asked where I learned the above about Gagné and Bloom.

Gagné’s Nine Events

Besides criticisms like Gagne’s Nine Dull Commandments, the post that really made me rethink Gagné was Tom Werner’s Whatever You Do, Don’t Drop Practice (now only available as an archive via the Wayback Machine). That post summarized research on what happens when you remove elements of instruction.

From Tom’s summary:

The researchers were interested in which of some of Gagné’s nine events of instruction were most powerful in promoting learning: objectives, information, examples, practice with feedback, or review.

The researchers pretested 256 college students enrolled in a computer literacy course and divided them into low, medium, and high blocks on the basis of the pretest scores.

They then divided each block of students into six groups and randomly assigned each group to a different version of an instructional program:

• Full program (objectives + information + examples + practice with feedback + review).
• No objectives (objectives + information + examples + practice with feedback + review).
• No examples (objectives + information + examples + practice with feedback + review).
• No practice (objectives + information + examples + practice with feedback + review).
• No review (objectives + information + examples + practice with feedback + review).
• Information only (objectives + information + examples + practice with feedback + review).

In a nutshell, each of the four groups that had practice with feedback scored significantly higher on a posttest than the two groups that did not have practice with feedback.

The study itself doesn’t specifically call out Gagné quite as much as this summary implies, but it certainly should make us pause before we insist on following that formula exactly. It also is proof that active practice with feedback really does make a difference. If a client asks for an information dump, this research should help support you in arguing for something including practice.

Bloom’s Taxonomy

At a previous job, we had regular quasi-formal professional development training for the instructional designers, provided by other members of the team. One person planned a simple game to reinforce Bloom’s taxonomy. The group was divided into two teams, and one person at a time from each team came up to the front and faced each other across a table. The “game show host” read a “Bloom verb” off an index card and the contestants slapped the table to see who could classify it first.

What would you guess happened? Think about a verb like “Determine”: where would you classify it?

The game almost immediately devolved into arguments over where the verbs belong. The poor activity leader had consulted a single list and didn’t even consider that different lists categorize verbs differently. Sometimes a single list classifies verbs in different places. This Bloom verb list, for example, classifies “identify” as both Knowledge and Comprehension; another list puts “compare” and “contrast” both in Analysis and Evaluation, depending on whether you use them together or separately.

How do you definitely solve an argument like that? Do you have research support for putting a verb in one category or another? Neither did Bloom. As far as I know, Bloom’s taxonomy was meant to be a theoretical framework and was not based on any sort of research. (If I’m wrong on this, please direct me to the research; I’d love to be corrected!)

This piece on Problems with Bloom’s Taxonomy asserts the lack of research:

The categories or “levels” of Bloom’s taxonomy (knowledge, comprehension, application, analysis, synthesis, and evaluation) are not supported by any research on learning. The only distinction that is supported by research is the distinction between declarative/conceptual knowledge (which enables recall, comprehension, or understanding) and procedural knowledge (which enables application or task performance).

That article outlines how the taxonomy is invalid, unreliable, and impractical, as well as offering alternatives more focused on performance.

Reality Check

I will admit, as I did in the interview, that I do still use “Bloom verbs” for writing objectives. I keep those verb lists handy because they help trigger ideas and focus on more active, higher level objectives. I’ve done objectives this way so long that it’s force of habit as much as anything else. I suspect I could get comparable or better results using the “Content by Performance” option in the article above. I’ve been in work environments that were really invested in Bloom, and I admit I’m not sure how much I’d fight that battle. If it’s a choice between fighting about Bloom or fighting to have realistic practice, I’ll choose to spend my efforts fighting for practice and context.

Gagné wasn’t emphasized as much in my education training, so I don’t have as much to unlearn there. The research above simply reinforces the need for practice with feedback. As the authors of the study point out, some of these elements may have value beyond improving posttest scores. Objectives can be useful in the development process. Personally, I want those objectives when I’m working with SMEs to help focus them on what’s most important, because that will improve the end result.

What about you? Do you use Gagné and Bloom, or have you rejected them in favor of something more relevant for your own work? Or am I way off base here, missing significant research that supports these ideas?

Image Credit: ? by Oberazzi