This page is powered by Blogger. Isn't yours?


Friday, August 23, 2002

The art of blending (cont)

Sure, Peter, that's fair.

I hear "Blended" most often from people who thought training-by-computer-alone was going to fly when explaining they've awakened to reality.

You thought I was going to take the bait and bring up the purity of single-malt training, didn't you?

posted by jay cross at 2:11 AM

The art of blending

Jay, would it be fair to say that much of what people claim to offer today as "blended learning" isn't so much blended (like a good Scotch whisky) but "tossed" (like a salad)? We need to distinguish between blended learning -- where there is a real input/output strategy and a dynamic structure -- and "tossed learning" where the form of input alternates. Maybe between the two there is "composed" ("salade composée": see picture 2).

P.S. I've borrowed the first picture from a site offering an education in whisky!
Distillery Destinations

posted by Peter Isackson at 1:31 AM

Thursday, August 22, 2002

One Way signThank you, Peter. You've nailed it. Lectures aren't bad per se. But relying on lectures to carry the full load of instruction is akin to building a house using only screwdrivers. Building a proper house requires hammers, saws, shovels, and all the other tools in the toolbox. Similarly, creating an effective learning experience requires discussion, application, enthusiasm, maybe a couple of hammers, and all the other methods at our disposal. The design challenge is to select the right tools to do the job well.

In short, academic lectures are not blended.

This Blog hasn't been blended either. It has been like looking through the two-way mirror into the interrogation room on Law & Order. Every week several hundred spectators listen but can't be heard.

DiscussionWe're going to change that. As soon as I get it coded, probably this weekend, the Learning Circuits blog will enable readers to post comments and questions. Participants will be able to request clarification and voice opinions during the Blog lecture.

posted by jay cross at 11:06 AM

Jay, I fully share your distaste for the one-way street lecture. What I find most amazing about lectures is that in the pluri-millennial tradition of developing, producing, testing, marketing, evolving the product we call the lecture, no institutions and only a few individuals have bothered to think about how the thing works and attempt to turn it into an art form, or at the very least into an efficient service.

It shouldn't be too hard to work out who should be lecturing, when, where, how often and to what effect. What is the added value of the live speaker? It can be rated on a scale of 0 to 10. If it is less than 4, forget it. If the lecture is an "event" (e.g. the revelation of recently conducted research), by all means do it. Curiosity and public communion justify it. It becomes a recognizable pretext and stimulus for highly public debate. By the way, good lecturers make every lecture appear to be an original public event (even if it is the same thing year after year), which justifies giving them a high rating on the scale of 10.

What environment is the lecture taking place in? If interactivity is built into the environment (or if the physical environment encourages interactivity), the role of the lecture can be defined and the impact of the interactivity predicted. Lectures can then have a strategy. If there is no planned interactivity and the only expected result is that learners will remember as much "information" as possible, put it on a website where people can access it, copy it, study it, integrate it into their research etc.

In short, if we think of lectures as public events aimed at stimulating exchange, we should end up with far fewer lectures than we have today, but ones that:
a) correspond to certain criteria of quality and will therefore interest people,
b) provide a stimulus for creative thought and deeper exploration of a subject.

The lecture, according to this reading, is a phase in a process. If there is no process and no method, it is worthless.

posted by Peter Isackson at 1:22 AM

Monday, August 19, 2002

If you're still scratching your head, wondering what this Blogging stuff is all about, read Steven Levy's Living in the Blog-osphere in today's Newsweek.

    ",,,with a new blogger joining the crowd every 40 seconds, Weblogs are officially the explosion du jour on the Net. Most estimates peg the current number at a half a million Weblogs...."

    "Blogging is a social phenomenon, and the Blog-osphere self-organizes into clusters of the like-minded. Within one of those clusters, the small-scale drama of a life, the incisiveness of one?s film criticism or the knowledge one imparts about esoteric telcom regulations can foment a weird kind of microcelebrity. "

When you've finished reading Levy, check out Newsweek's blog, The Practical Futurist. The article Cyber Scholars begins:

    Last week I returned to the university campus where I taught creative writing in the 1980s, this time to talk not about fiction, but digital culture. I didn?t know what to expect: during my teaching days, the liberal arts world didn?t exactly embrace the digital revolution. One writing professor I knew even refused to accept word-processed student papers?it degraded the prose, he insisted, and he could always tell the difference. I never had the heart to tell him that his students all wrote their papers on computers and then retyped them, just for him.

Pretty slick. Note the linked discussion forum, pointers to related info, and a place for the reader to rate the story. This looks better than a lot of eLearning I've seen on corporate desktops.

As to lectures, it strikes me that you two are apologizing for them, not defending them.

Peter, you enjoy the live aspect of lecture but clearly prefer two-way discourse to one-way lecture. You want a lecture that's not a lecture. Lecture comes from Latin legere, to read. It's one-way. Professor reads; you listen.

Clark, you welcome lectures by experts doing the same sort of things you're doing. Looking into their heads makes you reflect. Few lectures are like that. Where I went to school, most lecturers acted as if we students were empty-headed newbies.

This morning I attended the monthly meeting of eLearning Forum. Three speakers made presentations. Then members asked questions. Some were very pointed questions. The discussion was give-and-take. Interaction was encouraged, during the event and at lunch afterward. I found it a more satisfying learning experience than the traditional campus lecture.

posted by jay cross at 7:23 PM

Sunday, August 18, 2002

I'll take the opportunity to react to both of Clark's recent postings.

On the question of culture, I agree that we have to be active promoting cultural change and that does mean promoting values and encouraging new rituals. After all, as Jay insists, the new values emerge through the doing, not just the saying. The point I was making is that culture and its rituals will indeed evolve if they are encouraged, but may end up being somewhat different than what the "instiller" initially imagined. Culture has a way of defining itself, and if creativity is "tolerated", the end result may be even better than what we originally planned. So basically we agree. I simply worry that if instilled in too directive a way, the creativity may be stifled and the result appear artificial.

I also agree with you, Clark, in your defense of the lecture. It is a form of communication that still has its uses, even more so -- as you point out -- if an effort is made to integrate it as a component of a learning process that relies on other resources as well. The specific advantage of the lecture is the impact of the personality of the lecturer. By that I don't mean public speaking skills, though they are a key element in guaranteeing minimal reception. I'm referring to the fact that the lecturer delivers a discourse that is perceived through a speaking voice, with its human expression and personality. Unlike a text in a book or on a screen, the speaker is perceived as real and not a disembodied, pre-assembled sequence of abstract truths. By exposing his or her personality in public, the lecturer is implicitly inviting the listeners to situate the discourse at a human level, inviting criticism or contradiction. Learners still need to be aware of the tangible human reality of their teachers, the strengths and weaknesses of their personality and reasoning. This also allows learners to get a feel for the motivation for learning, discussing and expounding as well as to measure the visible limits of the lecturer's authority.

Physical presence is therefore necessary, at least occasionally, to situate the range and meaning of discourse. But of course lectures famously lack interactivity. The process would be complete if, alongside the lecture (but not necessarily synchronously), communication media allowed for some real exchange and reflection, rather than just the perfunctory questions that may or may not punctuate the end of the lecture.

The difference between discourse and information is the fact that in discourse someone is taking a risk and making a personal commitment to the message. The commitment and the message may be challenged by others. For certain topics, the people who develop or put forward original or complex ideas need to express them in public before committing them to writing. Until we find a way of duplicating the impact of lectures online (and videoconferencing doesn't quite do it), we won't achieve the same communicative alchemy.

Which is not to say that lectures which are mere textlike statements shouldn't be banished to an HTML page where they can be consulted at leisure rather than summoning avid listeners to a scheduled event.

posted by Peter Isackson at 8:54 AM

Thursday, August 15, 2002

Let me take the outrageous position of defending the lecture. In context. I, as a conference goer, often welcome the chance to hear a lecture. By a person engaged in activities similar to ones I'm engaged in, and reflecting on their processes and results. Particularly by someone reknown for their expertise and presentation. Since I've been actively engaged in practice, the lecture serves as a reflection (which is why even learning folks still go to conferences, even though supposedly they're not good learning experiences).

The problem with universities and other learning activities is that they seldom put in place activities for the learners with which the lectures are aligned and serve a useful function. So, lectures as an integrated component of a total learning experience can play an effective role.

That said, it's hard to imagine a case where the traditional lecture would be the most valuable way to use a knowledge resource like a professor!

posted by Clark Quinn at 11:03 PM

In response to Peter: Culture is not easy to create, let alone change. There always is a culture, but it may not be the desired one. Should one hope that it will change, or should one take action "against a sea of troubles and by so opposing, end them"? While designing a culture may not be truly possible, I believe that one can and should systematically subscribe to a set of values, communicate them, put in place policies that reflect them, and, yes, put in place rituals to support them. I'm reading Rapport's "Ritual and Religion in the Making of Humanity", and while not an easy read, it's clear that the behavioral signification of ritual is a significant component in value change.
posted by Clark Quinn at 10:40 PM

Wednesday, August 14, 2002

Today's New York Times carries a story, The College Lecture, Long Derided, May Be Fading, that begins, "One day in 1931, Hamilton Holt, president of Rollins College in Winter Park, Fla., startled his colleagues at an academic conference when he declared that Yale and Columbia, which he had attended in his youth, 'taught me virtually nothing.' The reason, Mr. Holt explained, was that the lectures delivered by his teachers, as with those delivered by professors almost everywhere, were examples of 'probably the worst scheme ever devised for imparting knowledge.'"

In my own case, I'm on record saying that I have learned a lot more from Google and Amazon than from four years at Princeton and a couple more at Harvard. Yet, as the Times points out, "The mass lecture, sometimes derided as 'auctioneering' or 'hog calling,' is still the dominant form of instruction at most major universities." Worse still, since virtually all training managers and instructional designers were indoctrinated by lectures, they unwittingly perpetuate the practice in the delivery of corporate training.

A university president said that "In the next 10 years ... the teacher ... will be more of a mentor and less of a didactic lecturer." She goes on to say that "In ... making huge amounts of information accessible, technology has, to a degree, obviated the lecture, which after all came into being to impart knowledge. Now that knowledge is at one's fingertips, pedagogy has focused on how to use that knowledge." Whew! I hope those mentors don't simply replace lectures with AOL accounts.

People learn by doing, don't they?

Do you agree or disagree? Within the next week, we plan to add the ability for all readers to post their comments to entries in this Blog. We don't want to appear to be lecturing.

posted by jay cross at 12:33 PM

Saturday, August 10, 2002

I just returned from a couple of weeks of delightful vacation with the family in Australia. (Look at the photographs if you're into such things, but cut up the volume or you won't hear me describing what's going on.)

My return was sufficiently recent that I'm still bathed in the afterglow of "beginner's mind," a state that leads me to question things I once took for granted. For example, in Australia the waiters, waitresses, bellhops, and taxi drivers are invariably friendly and helpful. Their employers give them a decent wage, so tipping is not expected. Hmm.... Why couldn't it be like that in the U.S.?

Another example, this time from eLearning. eLearning is not a cure-all. Blended? Blended is just a better definition of what eLearning should always be. Fact of the matter is, learning should not be our primary concern. You can have the greatest learning organization on the planet, but if it's not a doing organization, you are toast. Doing is all that matters. Doing is level four, the ultimate measure of success or failure, the creator of value in the economy, the justification for your organization's existence, and the source of your paycheck.

Learning is hardly the only pathway to getting things done. Knowledge, tools, and a personal network can get you there. Pedagogues tell me "Information is not instruction." I don't care. If information is sufficient to get the job done, let's go for it. Are we to give up learning from Google because it's not "instruction"?

Which brings me to Knowledge Management. Good KM often leads to more doing than eLearning. Learning fades; explicit knowledge persists. Marc Rosenberg pointed me to a down-to-earth article on knowledge management he recently wrote for Context magazine. I won't repeat Marc's advice here -- you can read the article for yourself; it's not long.

Describing the "Seven Myths of Knowledge Management," he warns against creating elaborate systems that fail to support doing.

The bottom line for learning, for knowledge management, for mentoring, for EPSS, for performance support, and on and on is always the same: getting things done.

Ironic, isn't it? I return from several weeks of feeding wallabies, absorbing Aboriginal art, and sipping tea, only to rant about doing as the be all and end all. Maybe I need a couple more weeks' vacation.

posted by jay cross at 12:38 AM

Thursday, July 25, 2002


I would submit that culture sometimes resists our attempts to "instill" it. Enduring culture, contrary to mission statements launched from on high, must take root before it can grow and have an effect on the environment. The challenge concerning the creation of a learning culture is first of all that of redefining roles and methods, but most of all, making sure they are actually deployed in a way the entire group can recognize. If this is effectively done, rituals, legends* and myths will become spontaneously associated with the new culture. It may not always be necessary to pre-plan them.

How is it that learning cultures are so difficult to find at a time when everyone concurs about the necessity of developing the "learning organization" (as if showing one's approval might be an effective way of bringing it into existence)? Learning means changing one's perspectives, rejigging relationships, reaching out in new directions, acquiring new strategies of adaptation through active and intelligent experimentation. A learning organization must do this on a major scale. Changing always translates by some form of loss: efficient but no longer productive ingrained habits are replaced by more potentially productive but not yet efficiently deployed new methods, in most cases more complex than the past ones. Unfortunately, management always seems to be more aware of the loss than the potential gain and is suspicious of change that doesn't produce immediate results, whose precise contours haven't been forecasted in detail, or that isn't predicated on the preservation of past forms of productivity. No one wants to pay for the time lost not only in learning but especially in adapting. And we mustn't forget that the standard model of productivity since the beginning of the industrial revolution has been that of repetition and replication.

Culture does indeed require rituals, but the rituals we try to invent and impose from on high may end up reflecting the fear of change more than its acceptance. The real challenge culture poses is multi-dimensional and focuses on accepting new relationships between people (groups, hierarchies) as well as new conceptions of time (learning time, producing time, social time) and resources (access, management, exploitation). I believe that if change really exists, both perception and behavior will evolve, generating meaningful rituals and myths. The evolution of myth and ritual can of course be managed, but is unlikely to be effective if dictated.

* Legends must exist before myths can assume any collective meaning. Legends are the narrative of remembered events initiated or executed by real actors. The significance of the acts is exaggerated in the repeated telling of the story, eventually deforming it into legend. The symbolic connotations of the recounted events crystalize into notions or relationships that are then enshrined as myths.

posted by Peter Isackson at 7:25 AM

Wednesday, July 10, 2002

Before setting off to create a culture, an interesting exerecise would be scan of trends. Here are a few


Source: Watson Wyatt

Source: Van Buren, M. & Erskine, W (2002). State of the industry: ASTD’s annual review of trends in employer-provided training in the United States. Alexandria, VA.

Source: Employee Benefits Journal, Volume 27, Number 1 March, 2002

posted by Margaret Driscoll at 10:13 PM

Monday, July 08, 2002

I was just at a special event being held at the Darden Business School. The topic was creating a learning culture. A major issue was how to instill values. I believe that an important direction to pursue is understanding myth and ritual. (This is the 'story' issue, currently so hot, carried to the next level. Not to undermine the importance of stories in learning, but to expand....)

Myths are, in my channeling of Joseph Campbell, the body of stories that explain our place in society (or culture, or sub-culture, read: organization) and the world. Further, ritual is a set of behaviors that people go through to cement their actions to values, and are tied into the myths that explain why those values are important. Fact: we design rituals when they don't already exist.

If we are going to successfully combine performance rewards with mission statements and values, we're going to have to transform our corporate stories to myths and link in a set of rituals that celebrate tranformations in the directions needed to go. Example from the mini-conference: a company that shoots a cannon, not when a mistake is made, but when the lesson from that mistake is understood. Now that's a ritual that celebrates learning!

posted by Clark Quinn at 9:35 PM

Sunday, June 23, 2002

Peter, here's another version of the value of expertise story, this one from my services page.

Our pricing philosophy.

You pay us a small fraction of the value of solving your problem. Up front, we discuss your needs and agree to a fair price for the project. We charge you for the doing the job, not for the hours or days we put into it.

If you are looking for spectacular results, you've come to the right place. If you're looking to get something on the cheap, we're probably not for you.

It's like the story of Picasso, when approached in a café by a beautiful woman.

"Please sketch my portrait, M. Picasso. I'll pay you fairly."

Lightening fast, Picasso draws six charcoal lines on his sketchpad, and hands the woman the sketch, saying, "That will be a million francs, Mademoiselle."

"A million francs! But, Monsieur, it took you but thirty seconds," she stammered.

"No, Mademoiselle," said Picasso. "It took me forty years to be able to do this."

posted by jay cross at 6:01 PM

Saturday, June 22, 2002

(Training) Time is Money (in the bank)

The following story, which I owe to Corrie Bergeron who happened to post it on the elearningleaders list yesterday, illustrates the point we're both making:

"It's the story of the master pipefitter who retired after thirty years at a chemical plant. A week later the superintendent calls him and begs him to come in and fix the main pump - no one can get it back on-line. The old gent walks in, looks the situation over, picks up a wrench and taps the machine. Instantly it begins running smoothly. He presents a bill for $10,005: $5 for tapping the pipe, $10,000 for knowing where to tap."

Here's a guy who knows what his training (formal and informal) was worth! And here's a company that's $10,000 poorer on the day he left.

And just to be clear on authorship and copyright (a sometimes conveniently neglected issue). I asked Corrie if I could quote him and he revealed that this was a story he had heard at a "time-management seminar back when rocks were soft". Author unknown.

posted by Peter Isackson at 1:22 AM

Friday, June 21, 2002

Peter, let me change the title of my book from Common Sense to Not Rocket Science. You and Margaret are of course correct that one needs a feel for finance and business to intuit cost/benefit. You also remind me of the multiple roles of ROI. Bill brought up focus and making well-reasoned decisions. Margaret makes the case that for training to play a strategic role, training managers must understand the firm's strategy. You add ROI as management tool, and present some thorny questions about timing, amortization, shelf-life, and write-offs. I concur with all of you.

So why is this Not Rocket Science? Because I was thinking about the most popular ROI situation discussed at training conferences -- the go/no-go decision on major projects that's made at the top.

Andrew Carnegie
Jay Gould
J.P. Morgan
John D. Rockefeller

What would Andrew do? What would Jay do? What would JP do? What would John D do?

Imagine one of these titans is reborn as CEO of a modern company. Here's how he might approach a decision to hire Saba, Outstart, Accenture, Oracle, SkillSmart, and SoftForce to assemble a $4 million eLearning system.

The staff would have shopped various projects around, gathered the figures, done due diligence on suppliers, run the numbers, assessed the impact of changes in the marketplace, and prepared terse summaries for each scenario. Six Velobound Business Cases for new investments rest in a pile on the coffee table at the CEO's weekend cabin. (This is going on simultaneously at the EVP’s place by the lake, the COO’s condo at Hilton Head, and a few other spots.) A couple of projects are no-brainers—they are so integral to the organization’s mission that giving a go-ahead is a mere formality. Projects that enter new territory, eLearning for example, warrant more detailed consideration. Eavesdrop on a top executive’s internal thoughts, you’d hear something like this:

    Good Heavens, this effort is going to cost us $4 million and change. But the skills and knowledge of our people are our hope for the future. It's getting chilly in here. The analysis shows that we’re already spending nearly that much on training. I wonder what Frank thinks. The ROI is better than building another fab plant but some of the underlying numbers are soft. Of course there’s no guarantee that the fab plant wouldn’t be another white elephant when it came on stream in three years. The breeze is picking up outside. I bet it rains tonight. Without eLearning, we’ll never become an eBusiness. Some of our systems are pretty creaky right now and would benefit from streamlining. I hope I don't have to replace the roof on the cabin this year. We need to shrink cycle times throughout our organization. We're bringing out products faster and faster. The sales force can hardly keep up. The customer support reps are swamped. This eLearning infrastructure would give Charlie a platform for broadcasting and reinforcing his message about transforming our organization. We can't take this on ourselves. Our IT staff would be swamped. This would wait in line behind the other mission-critical projects they’re working on. Years. I've got a new fishing rod in the trunk. Is the bait store open yet? More important, at least according to Geoff Moore’s Living on the Fault Line, is that keeping up with eLearning is not a core activity for us—we should outsource as much of it as we can. I wonder what Charlie thinks. The ballgame comes on in about ten minutes. I bet the bass are biting. Where do I come out on this one? I’m optimistic about the potential. It feels right. I’ll back it at the Executive Committee Meeting on Monday. eLearning, sure. I believe in it. I bet the bass are biting.”

Don’t believe it? Many senior executives have more faith in gut feel than in numbers. The numbers are input. The decision is broader. An Information Week survey revealed that “more companies are justifying their eBusiness ventures not in terms of ROI but in terms of strategic goals… Creating or maintaining a competitive edge was cited most often as the reason for deploying an eBusiness application.”

Making sound decisions is the rationale for calculating ROI. Often precision is not required. The back of an envelope will do.

posted by jay cross at 1:32 AM

Wednesday, June 19, 2002

Going beyond common sense

On the topic of ROI, I would love to agree with you, Jay, that it is just a question of common sense, but this is a case where common sense violates some hallowed traditions. Ignorance of basic economics, as Margaret implies, happens to one those traditions (cultures are defined as much by what they ignore -- or refuse to see -- as by what they affirm). Everyone seems to understand, though in the vaguest possible terms, the notion of investment: the immobilization of liquid resources for a hoped-for future benefit. The key to seriously managing (as opposed to convincingly asserting) as subtle a notion as the return on investment is integrating the notion of and defining criteria for amortization. When we try to apply this to training, we run into a few problems.

Who can give reasonable guidelines on how to amortize knowledge? Is the asset value produced by training measured in terms of direct cost alone? Does an hour of training always have the same value, independently of its cost? Should training in the specifications of a new product -- whose shelf life may be only one year -- be written off as a mere expense? Should training in management skills, which should last a lifetime and contribute to further personal development, be depreciated only over the full length of a career or a fixed mean duration (5 years, 10 years)? Or should it never be depreciated at all, but written off when the manager leaves? Should training of any member of staff who leaves the company be automatically analyzed in terms of costs incurred and immediately written off? And what about the impact of training on corporate culture and the collective body of knowledge, or even the overall "intelligence" of the organization? Where do those values reside? How much are they worth? What did they cost to produce? It is common practice to refer to them as the company's most precious assets; but can we quantify them? Can we estimate the replacement cost of lost knowledge? If we really want to know the value of training, shouldn't we try to take everything into account and then measure the identifiable gains against all known costs?

As far as I can see, the people in finance aren't willing to get involved in these issues by producing a model that the training people (or even top management) could effectively use. I agree with Margaret that the next logical step is to train the training professionals in the basic principles of corporate finance. But even if upper level executives increasingly see learning as a "strategic asset", will they one day expect their training managers to apply the rigorous methods of finance to training and produce reliable ROI analyses that can guide long-term policy?

Bill pointed to the purely rhetorical value of ROI, management using it "to thwart training for training's sake" and the training people using it "to win the trust of those who have the power to change the world". He also clearly demonstrates that value analysis is possible if we consider all the effects of training. The examples he cites are generally short-term, which means they are bound to be convincing without having to speculate about long-term gains! But as management tools, we could do even better. The point I'm making is that if we are really taking the term "investment" seriously, we should also be reasoning in terms of long-term ROI. To do that, we need to agree on conceptual tools more refined than balancing out immediate cost and short-term benefit. Where infrastructure investment is concerned, as is the case of many of the initial costs associated with eLearning (including training a first generation of trainers and administrators), this should be obvious. Indeed, the kind of investment needed to deliver training that is not only supported by technology but is also intimately linked to new forms of industrial organization requires this kind of analysis.

Does anyone, peering into his or her crystal ball, see this coming?

-- Picture: Tamsin Olivier in a dream sequence of "Welcome to Business" (co-production with France Telecom)
posted by Peter Isackson at 7:59 AM

Tuesday, June 18, 2002

State of the eLearning Industry
Report from eLearning Forum

It was the best of times, it was the worst of times.

In Menlo Park, an overflow audience listened to and discussed up-to-the-minute presentations on what's happening in the eLearning marketplace. Unfortunately, we got our wires crossed and treated a record number of remote participants to an hour or more of silence; that in itself makes a statement about the real state of eLearning today.

We began with news from recent conferences. Rob Harris found buyers attending Conference Board events in New York and San Diego much more sophisticated than ever before. I noted an upbeat, can-do attitude at Training Directors Forum but found ASTD somewhat glum -- few new offerings, some former major vendors missing in action, and reduced attendance.

Trace Urdan described a marketplace where vendors are out of sync with their users. Vendors are selling to line managers, but training managers are making more decisions these days. Buyers are often more savvy than the vendor sales people who call on them. Vendors no longer drive sales; buyers buy only what is critical in their businesses and do so on their own schedules. Buyers are looking for a single source of IT and soft-skills content, and increasingly they are finding it.

Vendor viability is an issue for buyers and will drive consolidation. Major buyers want to deal with stable players. A SmartForce may seem large in the eLearning world, but it's a drop in the bucket for a GM, which would prefer to deal with a PeopleSoft, SAP, or IBM.

Buyers also want learning management integrated with their ERP/CRM systems, a great opening for PeopleSoft, SAP, and Siebel. Buyers don't expect seamless, comprehensive solutons. An LMS-ERP combo that's 75% there is preferable to an LMS residing in a silo.

Tom Barron pointed out that venture capital is disproportionately scarce in the eLearning sector but some players have scored major refinancing nonetheless, among them KnowledgeNet, Powered, Outstart, Hyperware, Element K, Interwise, Pathlore, and Vuepoint. Some "outsiders" have entered the eLearning space, notably PeopleSoft and SAP which both announced new LMS in April, and streaming media platforms from Polycom, Virage, and Yahoo!

Tom sees several new marketing strategies at work. Vendors are forming alliances to lend credibility to claims of interoperability. They are focusing on hot business units such as sales and customer service. Verticals are becoming attractive targets, particularly financial services and regulated industries such as pharma, petrochemicals, and health. The illusive small business market is once again in the sites of ASP-based eLearning.

Eilif Trondsen described the international situation and the results of our member survey. In Europe, the UK is the largest and most mature market. The Nordic countries are sophisticated but small. Some German vendors have hit some bumps in the road.

Note: The PowerPoints and Survey Results are online.


Bill and Margaret, Are you two on target? Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes!, Yes! So who do you think is buying all those books about ROI? And why? If Tom Paine hadn't already used the title, it might be fun to write a white paper on the subject entitled Common Sense.

posted by jay cross at 8:41 PM

Getting Focus

I agree with Bill -- In many organizations ROI is about “demanding the training organization focus on the mission of the overall organization, not just the operations of their own department” What is interesting is that most of the courses, seminars, and books on this topic for training professionals assume people know the business basics that underlie ROI.

Unlike many lines of business, training managers often lack the basics such as understanding profit and loss statements, what contributes to the cost of goods sold, the impact of overhead, how investment business cases are made, and the expected rate of return on an investment. There will be greater focus on business basics for training managers as the budget for training & development become larger and more visible. In the future training mangers will have to be as skilled at managing the business issues as they are at managing the training issues. The focus on ROI will fade as training & development becomes a more integrated with enterprise strategies and training is perceived as being better aligned to key initiatives.

The ROI fixation is simply an indication that training is maturing as a support service and it is must engage the organization like other lines of business. This is bound to be a challenging transition because many training professionals need additional skills to assume the new role, upper level executives need see learning as a strategic asset, and there need to be examples that demonstrate this new role and its benefits.

posted by Margaret Driscoll at 8:32 PM

Friday, June 14, 2002

The dirty secret of why mangers calculate ROI

Hi, I'm Bill Horton, knowledge architect and rewiring specialist from William Horton Consulting. I want to hop onto one of the themes Jay has raised, namely ROI.

Several years ago, I worked with a client whose CEO had mandated ROI analyses from all departments. The client, a bright training manager with a track record of success, was in a funk. Try as she may, she couldn't come up with monetary measures of the effects of training. After all what is the value of a baby? Blah, blah, blah.

I called the CEO and asked him what he expected to see in the ROI analysis. He laughed and said he didn't care. "I just want to focus everybody's attention on the business." Most business and university executives know better than to base strategic decisions on a two-decimal-point difference in ROI figures. But they insist on ROI and other metrics as a form of business discipline to get get myopic unit managers to consider the mission of the overall organization, not just the operations of their own department. In the case of training, they use ROI to thwart "training for training's sake.".

ROI is not a crystal ball, a guarantee of success, or a ticket to a promotion. It is just a tool for a vendor or training manager to show that they have rationally and objectively considered the organizational benefits and costs of what they are proposing. And it is a test of whether the vendor or manager understands what the organization truly values. I've performed analyses where the "return" was not money but reduced time to market or more equitable training. The free spreadsheet for my book Using E-Learning at contains examples of a variety of goals for training and how they can be quantified.

The goal of ROI, as I see it, is to win the trust of those who have the power to change the world. To win their trust we must demonstrate we share allied values. And to do that, we must know what their real values are.

posted by William Horton at 4:48 PM

Thursday, June 13, 2002

Fuel for the Fire

A foreign correspondent asked for my take on the ROI of learning debates. My rapid reply:

    Most of the American training community is under the delusion that training has a freestanding set of measurements. I take issue with this. Business results are the only thing that counts in business training. Accounting is hopelessly obsolete, so that cloudies the picture, too. And every organization has not one set of metrics but dozens -- every stakeholder brings his or her own to the party. Finally, trainers and business managers alike focus on the short term to the detriment of the long. I've long taken the contrarian approach that learning is a long-term process, and system improvements thereto have lasting, in fact, cumulative benefits. Hence, the importance of meta-learning & learning to learn.

The correspondent wrote back, asking about the distinction between ROI and metrics. I explained:

    While timeframe is an important facet of this, so is one's position in the organization. Top management may value returns in terms of organizational agility and intellectual capital. Functional managers focus more on things like increasing sales and improving customer service. Training managers focus on levels of proficiency and certifications.

    Drawing by Vis-a-vis, Visual Storytellers, for Internet Time Group

    Team Competency Issues

    As to the issue Peter raises about the major unamed ERP software supplier, yes, measuring and training for team competence is a vital for corporations.

    At Internet Time Group, we expect the process of competency optimization will be the sweet spot in the learning/KM marketplace two to rhree years hence. (I say "we" because this draws upon research described in Ian Hamilton's and my new study on the five-year future of learning and intellectual capital.)

    By 2003, the functional driver and immediate objective of eLearning in the form of the performance-improvement process cycle will be workforce competence—the ever-changing proficiency requirements for workers to achieve essential job-performance goals in a fast-changing worker environment and market. Workforce competence must be regularly defined, and the difference between its actual state and expert levels managed through eLearning initiatives targeting those areas of job performance expected to translate directly into the achievement of business objectives and functional goals. A next-generation competency management engine able to model business needs and worker activities in rich and relevant detail, and apply effective inference logic to determine current, actual states of competence, will form the conceptual and logical “heart” of the future eLearning process cycle.

    Ian's and my take on the major issues surrounding workforce competence
    • Full modeling of competency
        Worker competence must be accounted for and "managed” in all its major facets.

    • The domains of workforce competence
        Each of the various domains in which worker competence exists must be understood and addressed as distinct areas with matching eLearning solutions.

    • Pragmatics of actual vs. expert competence
        There must be explicit, core logical functions by which the steps in the performance-improvement process cycle make use of competency data.

    • Making competency modeling practical
        The functionality of a competency management engine that realizes this equation and drives the steps of the performance-improvement process must be captured in its central pattern.

    • Business management through “competency passports”
        Finally, in order for the potential of this new eLearning paradigm to be embraced from the top of the business to the bottom, the relationship between managers who “manage” workforce competence and workers who “have and apply” competence must be crystallized in new terminology and practices that solidify managed competence as an organizational routine.


    A competency management engine organizes three architectural components:
    • Model. First, a “model” of workforce competence is organized and populated, and its correlations to workers, learning content, and business outcomes are established.

    • Infer. Second, each learner’s actual “state” of competence is assessed through a complex series of interfaces made from learning, job performance, and appraisal data gathered and analyzed from eLearning performance analytics.

    • Enact. Third, the competency needs of each learner are resolved through sequential learning actions that define improvement goals and the reasons for them, assemble the content to resolve them, and implement and personalize the learning to each worker.

    Peter, as for your three questions,
    1. Increasingly, my article of faith is "do what it takes to get the job done." If that means creating effective teams and helping them in whole or in part increase their competence, halleluia.

    2. The smart money is busy identifying the skills and roles requried to make this work.

    3. Should we be thinking differently...? Yes, this is one important item on a long list of important items.

    Too complex? I don't think so.

    Training not worth the investment? Maybe not, but the performance improvement is certainly worth the investment. It's high time we erase the line between training and other disciplines. Easier said than done, but shouldn't we all try to "Be what we can be"? Then again, perhaps I'm just an army of one.

    posted by jay cross at 5:58 PM

    Wednesday, June 12, 2002

    Now that the festivities in Cajun country are over (sorry I missed them), Jay invites us back to business. In recent days I've been talking to a major ERP software supplier concerned about having tighter management and delivery of customer training. They have become aware of the fact that the quality of training has a direct impact on the successful (or unsuccessful) use of their software. (Wouldn't it be nice if everyone could make the same discovery?) Failure or customer disappointment in the deployment of their product means no repeat business and poor testimonials. Letting outside suppliers (such as consultants) handle the training turns out to be risky, because the consultants aren't primarily intrested in training itself and even less in streamlining it.

    What I also found intriguing in the problem they've defined is that, given the desired operational end result, training must be targeted at a project team, not at a series of individuals needing to acquire mere procedural skills. One of the effects of successful training should be -- according to them -- the ability of teams of people trained together to work together. This remains true even if an eLearning solution is adopted.

    Can this interpretation of the training problem be the key to construct a model for future systems that combine human management of a group training process and eLearning modules? Building teams and the human skills teamwork requires -- even when the aim is to implement software -- must include some form of subtle human management with authentic human feedback loops alongside all the eLearning hoopla. What this ultimately means is finding efficient ways to associate high-level human training skills (many of which still need to be invented) and a library of appropriate eLearning resources. This leads me to formulate three questions (actually I can think of at least 36, but these are samples):

    1) Don't we need to rethink the shibboleth of "the individualization of the learning process" (as a basic desideratum or article of faith within a technology-based civilization)?
    2) Shouldn't we be busy trying to identify the new human skills and human roles required for this kind of model?
    3) Shouldn't we be thinking differently about the nature of the eLearning resources created for this kind of context?

    And the big question: am I wrong in believing in the feasibility of the model? Too complex, too expensive to deploy, not consistent with traditional training culture, impossible to organize and manage, training isn't worth the investment, etc.

    posted by Peter Isackson at 4:17 AM

    Sunday, June 09, 2002

    The ASTD Conference always draws a large international group. This year's contingent pinned their cards to a world map. The most well-represented countries were Korea (320), Canada (208), Japan (143), Brazil (136), and Denmark (91).

    I hope some of my fellow Learning Circuits bloggers are going to chime in here. I spent my last hours in New Orleans devouring another two pounds of boiled crawdads and taking one last stroll through the French Quarter.

    posted by jay cross at 7:43 AM

    Tuesday, June 04, 2002

    Ed Lawler, accepting a Lifetime Achievement Award, told us, "This is the golden era of human capital. Human capital management must be strategic in all organizations."

    Oscar Arias, former president of Costa Rica and recipient of the Nobel Peace Prize gave the morning keynote today. At least I think so.

    Five minutes of his presentation was all I could take. He was reading the text of his speech with no emotion. Slowly. I had a difficult time understanding him. What did get through was railing about immoral corporations. Hey, I'm against immorality, too, but I don't expect a few thousand sleepy ASTD members to do much about it. So I walked out and ended up spending most of the day at the Expo.

    The award for Most Arresting Booth goes to Achieve Global. They had hired a Belgian action painter who grabbed fistfulls of paint with his bare hands and smeared it on a 6' x 6' canvas. As he progressed, an image emerged.* It was an attractive show but I wonder if the metaphor is right. It said to me "Slapdash Development." Better than last year when Achieve Global's theme was a full-service gas station. That said to me, "We are obsolete."

    SimuLearn was giving demos of the simulation we've heard Clark Aldrich talking about for a year. The sim engages you in a conversation with from one to four avatars. The first five modules cover leadership; each module is 15-20 minutes long. You can insert your own conversations to customize the simulations; a full authoring tool will be available by year-end. Price is $250 to $500/seat. The cartoonish characters are driven by an AI engine. I'd expected a bit more graphic polish, but I'm generally over-optimistic on what technology can do..

    Art Bliss (left, drumming) filled the Expo hall with the sound of drums; tonight it was One World Musics turn.

    Most of the things I saw on the Expo floor were incremental improvements over what was here last year. There were few breakthroughs.

    TEDS told me about package called "Job Vision" that will add all sorts of functionality on top of their LMS. Imagine an integrated environment that incorporates employee resumes and career preferences, job definitions, requisitions management, job postings, resource locator, applicant processing, and performance evaluations. Sort of budget e-HR that integrates directly with learning management.

    Speaking of LMS, I asked one fiscally challenged vendor how they deal with questions about vendor viability. "No one has ever brought that up," they replied. I asked, "And what if they did?" The response: "We'd tell them to look at our next quarterly earnings report." I am so glad I don't work there. It's got to be tough. Bon Voyage!

    As a fan of visual learning, I enjoyed looking at some of the work of Root Learning. I'm sure this can speed time to productivity but might not magentic rubber shapes that people could use to invent their own scenarios be more powerful?

    Macromedia was showing off aspects of Studio MX, a suite of eLearning development tools (Dreamweaver MX, Dreamweaver Extentions, Fireworks MX, Flash MX, and a Cold Fusion generator.) Northeast Airlines demoed a tight little sim created in the new Flash. It looked quite handsome. (And saved Northwest $30 million by transferring skills training to the desktop. (Disclosure: Macromedia bought my dinner tonight. But Ive been using (and paying for) their products for years. Can any company lay claim to an equal breadth of products and experience? I can't think of any.

    Remember Teaching Machines? The Skinner pigeon box retrofitted for humans? Well, the apparatus in the photo is a teaching machine. The new kind. It teaches engineers how to program logic chips. The Feedback equipment enables one to set up miniature production processes. It was refreshing to see something so hands-on, akin to watching a family farmer plowing his fields.


    * This turned out to be Marlon Brando as the Godfather.

    posted by jay cross at 11:25 PM

    At a panel session, Clark Aldrich said we'll always need eLearning but we may not need today's eLearning companies. Tom Barron noted some big names that are not on the Expo floor this year: Oracle, Digital Think, EDS, Geolearing, Docent.

    Kevin Oakes described a future that the eLearning vendors would kill for. eLearning will become mission-critical, like customer relationship management (CRM). eLearning vendors are getting less than $100 a seat; CRM vendors sometimes pull in more than $1,000 a seat.

    Yesterday I fell under the charms of New Orleans and cut out early. New Orleans, for those of you who haven't visited, is reminiscent of Venice... mysterious, damp, decaying, exotic, beautiful, with great seafood to eat and a tradition of wearing masks. I hated to miss Conference sessions, but you must understand -- crawfish go out of season only a couple of days hence.

    Gotta go. Oscar Arias is giving the keynote this morning.

    posted by jay cross at 8:32 AM

    Saturday, June 01, 2002

    Jacking in from New Orleans

    Tomorrow the ASTD International Conference and Exhibition kicks off officially with Tina Sung ("Making the Leap to Success"). Unfortunately, some of the people who would most benefit from it will miss the keynote by Jim Collins, "Good to Great: Why Some Companies Make the Leap and Others Don't."

    You never know who'll you'll bump into in front of the Riverwalk.

    ASTD Directors Geary Rummler, Michael Lee, and Kevin Oakes

    In Mulate's, "the original Cajun restaurant," a bunch of us talk about knowledge objects, which LMS companies will survive, the essence of leadership, and the late Gordon MacKenzie over plates of blackened catfish.

    Ed Oakley, Bill & Katherine Horton chowing down at Mulate's

    Important news of the day. 1) It is hot here. 2) Crawdad season is almost over.

    posted by jay cross at 8:59 PM

    Here're some thoughts about what I hope goes on in New Orleans:

    First, I hope there're some discussions of the contexts in which people learn (more than 70% of which is not in formal situations). I've been thinking about it this way. This echoes Ellen's comments.

    I'll echo Peter and hope that the talk is about learning, not tools. The latter follows, not leads. I'd like to see more elearning companies with someone who really understands learning on their executive staff, as listed on their website.

    I also hope that people get beyond a knee-jerk reaction that there has to be engagement in learning, and start getting into specifics about just how to do it. Like usability in engineering, it can't be tacked on at the end, but has to be integrated into the design process.

    I hope that there's a lot of talk about moving beyond learning as a expendable component of business, and a recognition that learning is a fundamental component of innovation, the key to competitiveness.

    posted by Clark Quinn at 2:55 PM

    Friday, May 31, 2002

    Good morning,

    Here's what I've been thinking about as I consider what to pack for New Orleans.

    Over the past year, I've done a few presentations to ASTD local chapters.

    My presentations have been about the changing roles of training and development people, especially about the implications of emergent technologies and performance technology for the ways we do business. I wrote about it in Beyond the Podium and also in a piece for Learning Circuits, so you can see the gist of the message.

    Here's what baffles me. I thought I made compelling presentations. I tried to. I attempted to be clear about the shift beyond the podium to careers that manage knowledge more than deliver it, to the opportunities we now have for cognitive distribution, to new ways that instructors can serve in organizations, which typically involves doing less traditional 'instructoring.'

    Afterwards, in email and in conversations in my office at SDSU, I talked with some attendees about their thoughts and aspirations. It was as if they hadn't been in the room for my presentations. They were untouched by little I said about looming changes in our world.

    Our interactions sounded much as they would have in 1982. They liked to deliver classes and wondered about how to position themselves to offer workshops on communications (or teaming or sexual harassment) to companies and government agencies. When asked why they were intrigued with the field, they pointed to skills at delivering workshops and communicating with people.

    I'm not suggesting that those skills have no value. They do. For sure.

    It's just that they missed my points about the shifts in our field. I tried to make it vivid, to provide examples, to do things that would prod them to consider implications for their lives and planning, and still they see futures untouched by knowledge management, e-learning and systems thinking.

    Well, untouched is too strong a word. They just aren't moved and they are not shaping their professional growth decisions in those directions.

    That brings me to ASTD. The right people will be there and in large numbers, I hope. But are they READY to hear messages beyond how to break ice with participants?

    ASTD folks serve in the trenches. While we may be blogging away about social construction of knowledge and online communities and "ilities," that's just not top of mind with most people. Not even close, I fear.

    Will change come from the trenches? I don't think so. Will it come from leadership and their strategic vision? I wonder. Will it come because the products and services are so compelling that they sweep away resistance? I don't know. Do you think so?

    Now I have to decide what goes in the suitcase. Safe travels.


    posted by allison rossett at 10:36 AM

    I am looking forward to ASTD to look for the progress of all new content types.

    Communities, like this one;


    Coaching and other access to experts, facilitated by technology; and

    Embedded and even proactive help.

    posted by Clark Aldrich at 5:37 AM

    It's sometimes enlightening to see what people outside the business have to say and therefore suggest having a look at Lisa Napoli's MSNBC article on her own eLearning experience.

    She raises a particular point that seems to me worth examining in depth, concerning group discussion:

    "My problem with the discussion boards was the same I have with discussion boards of any sort: One or two people dominated them, and often used them as pulpits for digression, rather than as places to engage in relevant dialogue. Hendrick the teaching assistant did a good job of upbraiding the offenders, but the offenders turned me off to the forum itself."

    I believe there's a vast area that needs to be dealt with: distance group dynamics. The phenomenon she refers to is well known and of course we do know something about the psychology of Internet-based dialogue (discussion groups, chats, forums, etc.). One day we should be able to devise -- specifically for training projects -- sets of flexible techniques similar to those used for running a meeting, but with significantly different notions of time management, feedback strategies, organization, goal definition, interactive drama (where there is psychology there is always drama!), group and individual identity, etc. That's a handful to define, especially as the principles and techniques we are looking for can only come from validated experience. Also because contexts will always be wildly different (subject matter, duration of the project, number of participants, hierarchical levels, pre-existing relationships and other sociological considerations, modes of intervention) to say nothing of the variations in formality. Add to that notions of authority, hierarchy and unpredictable (or predictable?) phenomena such as spontaneous bonding and even possible instances of virtual mob psychology! Our work's cut out for us. Personally I find it as fun as it is challenging. But as Ellen Wagner has pointed out, business is slow to embrace the new thinking that is already well documented. How long will it take to show an interest in the key issue that group dynamics represents in the context of the "networked collaboration, knowledge management, communities of practices, learning woven into tasks, and other strategies" Margaret spoke about as been critical themes?

    posted by Peter Isackson at 4:37 AM

    Monday, May 27, 2002

    Isn't there an expectation that knowledge/human capital management is going to be this year's buzz? Or maybe just a low hum? I would think that this ASTD conference will be interesting in that the learning industry is in the midst of an industry-wide makeover. But I'll guess that the learning culture movement won't probably be that much in evidence yet - that'll be next year's ASTD conference theme.

    Dave Jonassen and I were chatting the other day about how odd it is that business is so slow to embrace some of the emerging new thinking about learning (e.g. activity theory, shared cognition, problem based learning) when those models reflect the reality of so much situated learning in the workplace. It's so much easier to focus on technology and tools.

    posted by ellen wagner at 7:41 PM

    Sunday, May 26, 2002

    I don’t expect to see many new technologies or practices because the innovations and changes this year are taking place in the “back-office”. These changes don’t have visible manifestation -- yet. For example, instructional technology has provided tools for collaboration, simulation, modeling, communities of practice, and knowledge management to support and extend learning. Despite the potential of these tools, they are not widely used and when they are used, they generally mimic a process or practice found in the traditional classroom. Significant changes in practice are required to take advantage of new technologies and those changes are evolutionary not revolutionary.

    The changes happening in technology won't be seen by the average ASTD conference participant because the changes are in the “ilities”. The term “ilities” refers to the jargon that often makes training professionals glaze-over … scalability, interoperability, and extensibility.

    The “ilities” are important because they increase the power of a software applications. Let’s be specific. Scalability refers for the ability to start with a small number of users of an application and to grow the a number of users as needed. If you have an e-learning solution that serves only one division it is somewhat limited. But, if you can scale it up, then you can extend the application to your worldwide organization. The ability to scale allows you to share knowledge with a broader audience and to create larger learning networks.

    Interoperability is the ability for two or more applications to exchange information and to use the information that has been exchanged. For example, you may have a human resource information systems (HRIS) and a learning management system, with interoperability you are able to pass information back and forth. This would allow the human resource department to store an employee’s “learner transcript” with the employees other HR data. Like wise, the training department could use data from the HRIS system to identify all junior sales people and to automatically register them for an advances selling skills course.

    Extensibility is the ability to add new features and capabilities to a software application. For example, a training department might have an learning management system that has a simple text based catalog. If the application is extensible, programmers can add functionality to the text-based online catalog to make it a HyperTextWindow catalog. Thsi extended program and enableslearners to click on the name of the instructor teaching the course and learn about the instructor’s background.

    The other back office activity – change management is also not going to be visible this year. The ability to realize improvements in the design, development and delivery of training requires significant change. Using technology to automate exiting processes is the first step in the evolution of learning from the traditional classroom to the networked organization. Organizations must pursue business process redesign, realign the training function with strategic objectives (more than lip service and some colorful PowerPoint slides), and embrace and adopt new learning models such as networked collaboration, knowledge management, communities of practices, learning woven into tasks, and other strategies that do not fit managements traditional view of “learning.”

    So, don’t expect visible change. Talk to people who are evolving, the changes are sublet. The ‘ilities” in the technology are moving on a parallel path with change management– and the underlying infrastructure is going to be there when organizations need it. I agree with Peter Isackson, I also want hear more about psychology and learning culture. More importantly, I want to hear about changing the mental models funders, visionary and empowered chief learning officers, reward systems that demonstrate learning valued, and business processes that align learning with strategic goals.

    posted by Margaret Driscoll at 9:00 PM

    Saturday, May 25, 2002

    "Depending on whom you believe, blogging is either having its 15 minutes of fame right now or these 15 minutes actually constitute the opening scene of a new thousand-year bloggian reich," writes Jonah Goldberg in Attack of the Blogs in The Washington Post.

    To help attendees at ASTD in New Orleans categorize vendors, we offer the Internet Time Group "Magic Quadrant."

    posted by jay cross at 10:29 AM

    Thursday, May 23, 2002

    It's easier to get 12 tall guys from Charlotte to New Orleans (we pass from bees in the previous post to Hornets) than one slightly smaller one from Paris, even though New Orleans is the birthplace of jazz. So, in reply to Jay's question, I can't say what I hope to "see" (and duly apologize, but promise I will make the effort in 2003).

    I can say what I hope to hear: more about psychology and learning culture and perhaps a little less about technologies and techniques (which is not to belittle their importance). When the debate starts focusing on people's states of mind, values, expectations, perceptions, both as e-learners and e-trainers, it will be a sign that eLearning is maturing enough to become part of the mainstream.

    This is not, by the way, meant to be a contradiction of Tom's contribution, but rather a complement to it.

    posted by Peter Isackson at 1:47 PM

    One thing I think we’ll see is a growing number of authoring tools that makes use of SCORM and “learning object” concepts. A recent survey we conducted with Learning Circuits (see Learning Object Approaches Making Inroads) indicated a strong interest in learning object approaches to building and managing content AND a strong interest in custom-developed content. New tools help automate or template the process of building content in modular formats using instructional design principles. We’ll also see more, and broader, integrated offerings that answer current woes around integration and interoperability—through alliances among LMS, LCMS, synchronous tool and content vendors, including more co-hosted exhibits. LMS demos and discussions won’t dominate the show as they have in the past--I predict a renewed focus on content development and management. We’ll also see some interesting new tools that automate development of learning and performance support content around computer-based processes, such as ERPs. And another new category of tool that might be found on the fringes focuses on managing and searching metadata that is wrapped around SCORM-conformant content.
    posted by Tom Barron at 9:25 AM

    Wednesday, May 22, 2002

    If you're joining us for the first time and not familiar with blogs, welcome! You might want to check this article from Learning Circuits or this page at Internet Time Group for background.

    The ASTD International Conference is coming up soon in New Orleans. For those of you who can't make it, I plan to post daily observations here. And, as in previous years, I'll include a few snapshots.

    A question for my fellow bloggers: Do you expect to see anything new at ASTD this year? What? What would you like to see?

    posted by jay cross at 9:44 PM

    Monday, May 13, 2002

    Curiously, Clark and I don't seem to know for certain whether we "furiously" agree or not. This seems rather typical of the whole learning business. I tend to agree with Clark that we do agree! The problem is that at different times we are probably referring to different phenomena. My suggestions were very general, pointing towards the overall strategy for handling a variety of content, which I see as process (transforming input into output). I also glanced at questions of content selection in the light of cultural variation. When we focus on specific content needs, particularly the "learning objects" we hope to find somewhere or need to produce ourselves, we are faced with these cultural problems, which, as Clark points out, constitute helps or hindrances depending on 1) the profile of the individual learner 2) the trainer's awareness or even real knowledge of that profile. I think a lot of work needs to be done on both at the same time. I don't believe we have any valid human models yet for dealing with this efficiently (i.e. converting information into effective strategy) and everyone else (i.e. the knowledge management specialists) seems to be focused on structuring the information. I believe that this is only the first step and may need some guidance from the strategy side to develop the right structural models.

    A new theme occurred to me today and I have no idea what it's worth or how far it can be taken, so for the sake of my own ongoing reflection I'll state it here (I need to set it down somewhere!) and await any constructive or, why not, destructive criticism. It is curiously linked to the bee in Clark's bonnet, but inverted (the stinger is on the other end!). The notion has to do with the teacher’s or trainer’s state of knowledge -- not the learner’s -- before and after a course. I am not, however, suggesting pre- and post-testing! I am suggesting that it should evolve, almost as much as the learner’s state of knowledge and that we should take an interest in tracking this evolution. The context I am referring to is that of collaborative online learning. This wouldn’t be the same thing for traditional face-to-face teaching (but see my final remarks below), and even less so for pre-programmed eLearning (which I see increasingly as isolated or modular learning objects, whose meaning and impact derive from the variable contexts in which they are used more than from their internal merits).

    My notion is that of a kind of open or “improvisational teaching”, a strategy that specifically aims at learning to teach a particular course by teaching it, after defining its overall structure and logic. It proceeds from two observations:

    1) no one can fully anticipate what will happen in the learning process, particularly in distance learning,
    2) we do not necessarily know in advance what resources, among all that are available, will prove the most productive for real learners (in all their cultural variety).

    My notion of improvisation is borrowed from jazz, one of my previous occupations*. To be good at improvising, you have to learn not only the art of soloing (which you at least partly invent), but you must also know the chord changes (+ variations) of the tunes you are playing, the chosen style for each number, your precise role in the ensemble sections and, especially if you are accompanying rather than just soloing, have a good idea of the style and system of each of the other players. These multiple constraints nevertheless leave you free to discover through playing the things that work and don’t work both in general and specifically with regard to each type of musical event. The most interesting thing about working with other musicians is what you learn from them each time you rehearse or play. And of course the more you play a particular tune, the easier it gets to keep it going and to find ways of innovating and surprising without upsetting the underlying logic and the other musicians.

    In short, I’m in favor of under-planning one’s course strategies and leaving room to for us to learn from the learners themselves. Actually it’s less under-planning than avoiding over-planning. This means, without sacrificing one’s “authority”, learning how to encourage the learners to bring things to you (discovery of appropriate resources you may not have been aware of, new ideas or ways of looking at the material, patterns or sequences of behavior that produce learning more effectively than your initial game plan). In other words, we should seek to be instructional co-designers rather than instructional designers.

    It might be said that what I’m describing is a form of beta testing. But its implications are very different. You beta test something that is fully designed down to the last detail. What I’m suggesting is a system in which we as trainers and designers are actively concerned, at least the first time around, to integrate elements that come from the learners, or rather our own interaction with the learners. This can obviously only apply to collaborative training. But it can lead to strategies for producing learning objects. Much needs to be said on how to conduct this approach (how to create the overall model, how to manage events, how to communicate with learners, how to react to embarrassing mistakes, how to make permanent or replicable everything one learns, etc.).

    After a brief search on the web, I found that David Hammer of the University of Maryland, in a context of traditional face-to-face instruction, calls a similar approach “discovery teaching” and identifies some of the areas of resistance to it by teachers. My contention is that it is less risky and more appropriate in an online environment. It is also easier to structure, plan and capitalize on.

    * I ended up living in Paris because, after participating in a free-for-all jam session organized by Steve Lacy at the American Center nearly 30 years ago, I was offered a permanent job as a pianist (accompanying dance classes at the Université de Paris) and accepted it in order to become fluent in French!

    posted by Peter Isackson at 2:51 AM

    Saturday, May 11, 2002

    Well, I really want to reply to Peter, but right now I've got a bee up my bonnet, and I want to vent (how's that for mixing my metaphors?). I'll get to Peter's comments in a moment...

    In recent work, I've reliably been coming up against a requirement for a pre-test. And I can't for the life of me figure out why, they're not using the data to do anything but compare it to the post-test! This didn't make any sense to me, so, I did a Google search to see what came up. In "Going Beyond Smile Sheets... How Do We Know If Training Is Effective?" by Jeanie McKay, NOVA Quality Communications, I came across this quote:

    [Level Two] To evaluate learning, each participant should be measured by quantitative means. A pre-test and post-test should be administered so that any learning that takes place gets attributed to the training program. Without a baseline for comparison of the as-is, you will never be able to reveal exactly how much knowledge has been obtained.

    Now I don't blame Jeanie here, I'm sure this is the received wisdom, but I want to suggest two reasons why this is ridiculous. First, from the learner's point of view, having to do a pre-test for content you're going to have to complete anyway is just cruel. Particularly if the test is long (in a particular case, it's 20 items). The *only* reason I can see to do this is if you use that information to drop out any content that the learner already knows. That would make sense, but it's not happening in this case, and probably in too many others.

    Second, it's misleading to claim that the pre-test is necessary to assess learning. In the first place, you should have done the work to justify that this training is needed, and know your audience, so you should have already established that they require this material. Then, you should design your post-test so that it adequately measures whether they know the material at the end. Consequently, it doesn't matter how well they knew it beforehand. It might make sense to justify the quality of the content, but even that's falacious. We expect improvement in pre-post test designs (this is forbidden in psychology as a mechanism to determine the effectiveness of an intervention, without a control group), so it doesn't really measure the quality of the content. Though it could be considered a benefit to the learning outcome, there are better ways to accomplish this. There is no value of the pre-test in these situations, and consequently it's cruel and unusual punishment for the learner and should be considered unlawful.

    OK, I feel better now, having gotten that off my chest. So, on to Peter's comments. I agree that we want rich content, but if we have the current redundancy to address all learners, we risk being boring to all to make sure everyone's learning style is covered. We *could* provide navigation support through the different components of content to allow learners to choose their own path (and I have). That works fine with empowered learners, but that currently characterizes no more than about half the population. The rest want hand-holding (and that's what we did), but that leaves the redundancy.

    Which, frankly, is better than most content (although UNext had/has a similar scheme). However, I'm suggesting that we optimize the learning to the learner. I'm not arguing to assess their cultural identity, but to understand the full set of capabilities they bring to bear as a learner (my cultural point is that we're better off understanding them as individuals, not using a broad cultural stereotype to assume we understand them). That is, for some we might start with an example, rather than the 'rule' or 'concept'. For some we might even start with practice. We might also present some with stories, others with comic strips or videos. Morever, we drop out bits and pieces. A rallying cry: Use what we know to choose what to show. Yes, additional steps in content development are required to do this (see my IFETS paper), but the argument is that the payoff is huge...

    The assessment is indeed a significant task, but in a long-term relationship with the learner, we can do something particularly valuable. If we know what their strengths and weaknesses are, as a learner, we can use the former to accelerate their learning, and we can also take time and address the latter. A simple approach would be to present 'difficult' content with some support that, over time, would be internalized and improve the learner's capabilities. Improving the learner as a learner, now THAT's a worthwhile goal!

    I strongly support Peter's suggestion that using a rich world as a source for embedding (or extracting) learning to make it meaningful is ultimately valid, and the base of much of my work on making learning engaging. We may be agreeing furiously, except that I may not have made what I meant by learner assessment clear.

    In answer to Peter's query, I'm sad to report that we have not, and can not, publish on the 31 dimensions. I can only suggest the path we took: using Jonassen & Grabowski's Handbook of Individual Differences as an uncritical survey of potential candidates, as well as other likely suspects from any other source your research uncovers, then to make some sensible inferences to remove redundancies (much as the 'Big 5' personality factor work attempts to make sense of personality constructs). Make sure you cover the gamut of things that might influence learning, including cognitive, affective, and personality factors.

    posted by Clark Quinn at 5:01 PM

American Society for Training & Development (ASTD)
© Copyright 2002 | Disclaimer | Privacy Statement
1640 King Street, Box 1443 Alexandria, Virginia, 22313-2043 USA
Phone: 703.683.8100 - 800.628.2783 - Fax: 703.683.1523