Another Pro Bono Day

Need a white paper on eLearning? A stirring sales presentation? A new marketing concept? An introduction to an important contact?

Internet Time Group has capacity; give me a call.

Capacity. That's why I devoted the day to pro bono work. First, a no-fee presentation on the state of eLearning in Silicon Valley and the opportunities for doing business there. Second, responses to a Portuguese journalist preparing an article on ROI and meta-data. Here's my reply. If you have another take on things, please make a comment.

"Here are my rapid-fire answers to your questions. Please email me a copy of your article when it's printed.

1. How do you evaluate training in the e-learning paradigm?

eLearning is no different from other corporate expenditures. You evaluate its success or failure in meeting the business objectives you set out for it.

2. ROI for e-learning has been a dominating issue. In you view, what are the appropriated metrics for measuring ROI for learning?

ROI is another way of saying cost/benefit analysis. You assess the benefits you received (usually things like higher sales, better customer service, speedier processes) and compare them to what it cost you (mostly the opportunity cost of the people learning but also computer, administration, and license fees).

3. What are the main difficulties in determining results
of training (its intangible factors) in financial terms and their measurement? Is it hard to link it directly with training? Why? Does the adoption of e-learning standards have any effect in this process?

The difficulty I come upon most often is poor up-front planning. You set your goals in advance. You gain your internal sponsor's agreement on your logic, that is, if we do this, we expect that, and that's an indication of this result. It doesn't have to be scientific; rather, it has to convince management.

eLearning is no harder to measure than other intangible items. What's the value of advertising? What's it worth to be more careful in hiring? The only effect of standards is cost, the "I" of ROI. At first, adhering to standards will cost more than doing nothing. In time, factors such as reusability, modularity, and interoperability will lower costs. Dramatically.

4. And how can e-learning increase ROI for training?

When used appropriately, eLearning provides more learning for less cost. It's often more convenient, and therefore more likely "to take." It is sometimes faster, for example, providing compliance training to tens of thousands of people in a matter of weeks, something you simply cannot accomplish with classrooms and instructors.

5. How does ROI demonstrate the monetary benefits of the cost of training?

?? I don't understand your question.

6. Part from the ROI evaluation is based on the opinion of the stakeholders. How is possible to overcome this subjective
dimension and obtain accurate, unbiased information?

Whoa! Subjective doesn't mean inaccurate or biased. It's important for stakeholders to be actively supportive. Sometimes numbers like ROI get in the way of common sense.

Some people think of ROI as just a number. That simplifies things at the expense of knowing what's really going on.

7. But there is fear of negative ROI - many do not want it, proven that a training program is actually bad for the company. How to overcome this?

Some training and eLearning is probably not worthwhile. I've certainly seen lots of eLearning that was a waste of time and money. That's one reason for assessing ROI, to identify and eliminate the programs that are not working -- and to improve the others over time.

8. Going to the issue of e-learning standards - how can
the adoption of e-learning standards increase the accuracy of ROI for training?

I consider standards and ROI separate issues. Standardization in general makes things uniform, and therefore easier to count. At the same time, it may obscure important but subtle distinctions.

9. In which ways e-learning meta-data facilitates a better measurement of the transfer in learner?s behaviour?

I don't think it does.

10. How can e-learning meta-data contribute for identifying the specifical learning profiles of each trainee?

Meta-data could be coupled to a competency management system to identify gaps in knowledge -- and objectives for future learning. I recently did some research in this area. Lots of people talk about this, but it's always something they'll do next year. Sort of like the economic future of Brazil, which has been just around the corner for centuries now.

11. Are there other advantages that can occur from the
adoption of standards in the e-learning meta-data?

Meta-data standards are the key to interoperability. They are the Rosetta Stone that makes learning experiences from many sources work together seamlessly.

Just as XML and a set of related protocols are the foundation of "web services" and the Semantic Web, meta-data standards are a significant part of the underpinning of a universal system of learning.

12. How do you analyse the content market in USA? Is it conformant to e-learning standards? What about European market?

The military is ahead of everyone, but of course they can enforce whatever they want and also have the money to pay for it. On the corporate side, both in the US and Europe, standards are just beginning to take hold. Don't hold your breath waiting to see everyone jump on the standards bandwagon.

13. Should organizations have a common repository for all content, and if so, which rules will govern how the system is used?

All organizations? Never. There are economies of scale at work here. Tiny organizations will be better off with apprenticeship than with eLearning for a long time into the future.

The question of a common repository is another instance of the perpetual battle between centralization and decentralization. Logically, centralization is more efficient. The problem is that many gigantic systems topple under their own weight. Consider Enterprise Resource Planning (ERP). Brilliant software, very expensive, lots of hard work, and yet half the time it fails miserably.

If things are truly interoperable, rules won't make much difference, will they? There's not much difference between sharing and owning content.

14. Will any governance structures be needed to help ensure adherence to standards within and organization?

All standards require policing: identifying those not in compliance. I think the market mechanism, the "Invisible Hand," will reward those in compliance and drive the others from the market.

Who's going to buy non-standard material in a world that has agreed upon standards? It would be like buying a 7" CD: there's no way to play it.

Posted by Jay Cross at March 3, 2003 10:40 PM | TrackBack