These are excerpts from Metrics, an eBook available in the store.
Learning about ROI seems to be enjoying a renaissance in the training industry. Workshops and certificate programs abound. However, the courses I've looked at teach things that no business manager would buy. Here, let me tell you why I feel that way.
Metrics are measurements that matter. The Industrial Age is over. Measures that fail to account for intangibles are misleading.
Decision-makers use metrics to
Metrics are in the eye of the beholder. They are not simply the application of a rote formula or accounting rule. They are subject to interpretation. This is what makes metrics worthy of discussion.
Training jargon doesn't play well in the executive suite, so you need to express yourself and position what you bring to the table in business terms.
If only I had $10 for every time I've heard training managers lament that they can't separate out the impact of the training from everything else that was going on. Some suggest that certain employees go untrained to provide a control group. (Forget it; the Hawthorne effect* would skew the results.)
|*In a classic experiment in the 30s at Western Electric's Hawthorne Works, researchers found that workers were more productive when they cut the lights up. Also, when they cut the lights down! Conclusion: Workers are more productive if you pay attention to them. Placebos work.|
Why would you want a control group anyway? Business is not precise. Deciding whether to invest in more training or increasing bonuses is not some physics experiment requiring 6-place accuracy. Consider John Wanamaker's regret, "I know only half my advertising is effective. If I only knew which half." Wanamaker didn't become a department store mogul by cutting his ads; he did what his gut told him to do.
Who decides whether an iffy investment, like Wanamaker's ads or your training program, is worthwhile? Your sponsor. The sponsor is the person who most strongly influences the decision on how to spend the money. The sponsor is your client. The sponsor decides what markers constitute proof.
You've got to describe the linkage of your initiative and business results quantitatively, using assumptions your sponsor will buy into. You must be explicit about the what-if's. Do this in writing, as a "Performance Agreement" that:
The Agreement also shows that you understand the business and that you're on the same page as your sponsor.
Before you get too far into metrics, ask yourself, "Does it matter?"
One of the few aspects of accounting that I like is The Principle of Materiality. This principle says that if it doesn't matter, don't worry about it.
For example, if Chevron-Texaco?s accountants uncover a $32,000 error in the sales department?s expense budget, they don't make Chevron-Texaco note the error in its annual report. Chevron rakes in $100 billion a year. $32,000 is a drop in the bucket; it's immaterial. Now then, if the accountants find a $32,000 discrepancy in your personal expense report, that's material. Send us a postcard from the slammer.
You can?t measure everything. Therefore, you should seek to measure important things. Let everything else coast. Don't fritter away time on the small stuff.
While training directors may have different objectives from CEOs, everyone in today's business world shares one need: they want it all now. Benefits you don't see for two years are hardly benefits at all. Given enough time, a million monkeys at a million terminals could develop your entire curriculum, with Flash animations and a repository of SCORM-compliant objects. Nobody's got time to wait.
An appropriate metric for most eLearning is time-to-proficiency. How long will it take until your people are performing competently? By competent, I mean able to meet or exceed the expectations of customers, be they internal or external to the organization.
ROI is often a mask for uncertainty or an attempt to quantify cost/benefit with accounting principles that don't count people as assets. The business return on eLearning investment should be so obvious that you can figure it out on the back of a napkin.
Traditionally, executives assume training has little or no impact on revenue, so they measure training benefits in terms of cost savings. This works against eLearning, where increases in top-line revenue generally exceed reduced expenses by a wide margin.
ROI or cost/benefit analysis is relative, not some absolute value like the speed of light used to be. Where you stand depends upon where you sit. CEOs don't care about learning objects or LMS. Line managers focus on the performance of their unit, not the overall corporation. Training directors don't allocate resources to business transformation. One size does not fit all.
Present-day accounting is an anachronism. Invented half a millennium ago to maintain accurate shipping records, double-entry bookkeeping helped Venice dominate its part of the world. Formal accounting worked well when you could go out to the warehouse to count your assets. In the information age, it's an inappropriate yardstick for measuring anything. Most assets drive home every night.
In a nutshell, the basic problem is that accounting recognizes nothing but physical entities. Intangibles are valued at zero. Vast areas of human productivity -- ideas, abilities, experience, insight, esprit de corps, and motivation -- lie outside the auditor's field of vision.
Again and again, I've found the largest overall cost of any corporate learning endeavor is the cost of people's time. I'm not talking about salaries and benefits; I refer to the value they would have created had they not been tied up in training. Opportunity cost per hour is not a fixed amount. A salesperson's time during working hours in peak buying season is worth much more than the same individual's time after closing time in non-peak season. eLearning often enables the employee to shift learning to those non-peak hours.
I could go on for another ninety pages. In fact, I do just that in a newly published eBook titled Metrics.
30 Poppy Lane
Berkeley, California 94708
1.510.528.3105 (office & cell)
Subscribe to this Blog
Our Infrequent Newsletter
Entries by category...
Recycled from Blogger
Internet Time Group
© 2004 Internet Time Group
Blogger Experience, Housekeeping, Something New
Demographics is destiny
Are you setting the bar high enough?
Work as Video Game
Oracle and Macromedia, Sitting in a Tree
ASTD Silicon Valley
Kingsbridge Conference Center
First Post by Email
RSS Feed for New Site
Testing ... testing ... 1...2..3
IT Doesn't Matter - Learning Does.
All blogging is political
Damn, damn, double damn
The New Religion
Wolf! Wolf! Wolf! Wolf! Wolf! Wolf!
Business Process Management (2)
Business Process Management Conference
Don't Lose a Common Sense: LISTEN
It's only natural
Go with the flow
Time Out for the Fair
Informal get-together in SF this Wednesday
Repetition, reverb, and echoes
Push vs pull
The Big Picture on ROI
New Community of Practice Forming
Training Directors Forum 2004
A Rare One-Liner
PlaNetwork LIVE 2
ASTD 2004 Leftovers
Worker Effectiveness Improvement, not KM
Jay's Talk at ASTD
Mintzberg & Cooperider
Lest ye forget
ASTD International Conference & Exposition 2004
What is Workflow Learning?
ASTD msg 1 of n
Look out, it's Outlook
Collaboration at ASTD Next Week
Tell me a story
The shortest presentation on metrics you will ever hear
Back to Blogger
The Alchemy of Growth
Very loosely coupled
E-Learning from Practice to Profit
Robin Good kicks off Competitive Edge
Emergent Learning Forum: Simulations
The Best Things in Life Are Free
Metrics and Web Services
OpEd: ROI vs. Metrics