OpEd: ROI vs. Metrics

Learning Circuits just published my article, ROI vs. Metrics, which I thought was just telling it like it is, but editor Ryann Ellis considered sufficiently controversial to make it the first OpEd piece they've ever run.

What's controversial with this?

ROI is often a mask for uncertainty or an attempt to quantify cost-benefit with accounting principles that don't count people as assets. I contend that the business return on an e-learning investment should be so obvious that you can figure it out on the back of a napkin. Traditionally, executives assume training has little or no impact on revenue, so they measure training benefits in terms of cost savings. This works against e-learning, in which increases in top-line revenue generally exceed reduced expenses by a wide margin. Enter metrics.

I don't understand how anyone could disagree when I write

Present-day accounting is an anachronism. In a nutshell, the basic problem is that intangibles are valued at zero. Vast areas of human productivity--ideas, abilities, experience, insight, esprit de corps, and motivation--lie outside the the purist's field of vision.

A friend of mine read the article, came to this site, and plunked down $250 for the latest version of Metrics. If his $250 subscription to Metrics helps him justify his $500,000 eLearning budget, it's cheap at the price. If his newfound attitude improves his career options, that's icing on the cake.

The next version, Metrics 2.0, will be a great improvement, but it's not out yet. Here's a piece from the introduction:


Last year at TechLearn I promised to send several people my thoughts on measuring the value of learning investments. On the flight home, I assembled a batch of white papers, interviews, and articles I’d written over the past few years and was surprised to find I’d come up with 90 pages! This was more like a book than a sheaf of white papers.

Why had I written so much on learning metrics and ROI? Frustration. I’ve been in the training business for nearly thirty years, but before that I had been a systems analyst and market research director and I’d earned an MBA. I cut my teeth selling loan officer training to major banks. That requires real what's-in-it-for-me ROI. It pained me to go to conferences like TechLearn and Training and ASTD, only to hear the same worthless claptrap about Level 1 and Level 2 and ROI. Many of the “experts” got their expertise from textbooks rather than the real world. This made me angry. I wrote to expose the charlatans.

Metrics 1.0 & 1.1

Metrics 1.0 was little more than the white papers and articles put in a logical sequence. I slapped on a table of contents and inserted transitions. I sent copies to two dozen friends in the business for advice and comment.

I love books. Several rooms of my house are lined with bookcases. Last year I donated eight cartons of books to the library to clear some shelf space. Already, the gaps have filled in, and the shelves are packed. Nevertheless, I chose to make Metrics an eBook rather than a printed one. Our understanding of how to measure value is in flux; printed books become dated so quickly. I also figured that if I responded to readers’ questions and advice, the work would always be getting better. So Metrics 1.1 was issued as an eBook – and I promised purchasers they would receive the next version as well as this one.

Metrics 1.2 & 1.3

Metrics 1.2 was considerably tighter. I incorporated a mind map to introduce the subject and a simple flowchart for carrying out the full cycle of the Metrics process. I corrected glitches readers had pointed out. Version 1.3 was a minor upgrade, mainly error correction. Several readers pointed out weak spots in the manuscript. They wanted a more direct, forceful, organized presentation. They also asked me to spend less time trashing the current situation and more time on what to do next. To get Metrics into distribution, I offered it to charter subscribers for a mere $25.

Few readers actually gave much feedback but one individual made up for the silence of the rest. The GAP’s Dave Lee went beyond the call of duty. Dave’s background in publishing, experience in accounting, and current work in eLearning make him the ideal critic. Dave is a major influence on the improvements you’ll see in version 2.0. He offered suggestions on overall organization. He convinced me to take a more positive attitude, for example, telling me, “Jay, I’m not sure what accountant ran over your puppy when you were young, but you really don’t need the strawman of 'accountants are out to get us' to make your argument.”

Metrics 1.3 is the current version. People who purchase it will receive Metrics 2.0 when it is ready. The charter subscription period is over. The price of Metrics 1.3+2.0 is $250.

Metrics 2.0

Metrics 2.0 is a total reorganization and rewrite of the original material. It focuses on what to do and why in lieu of me bitching about the ramblings of false prophets. I’ve chopped superfluous material and added more explanatory text. I expect the final Metrics 2.0 to be ready in a couple of months.

People tell me they buy Metrics because they have an immediate need. They’re in budget trouble; their management just doesn’t get it; the big boss wants to see the numbers. In all likelihood they already know part of the material and have come to me to fill gaps or help them polish their approach. This latest version of Metrics begins with a roadmap of what’s to come. The map will guide you to the chapters that contain what you most need to know. I’ve relegated more philosophical issues to the back of the book.

Making the business case involves a lot more than doing the math. You have to understand the business. This takes credibility with managers outside of the training function. You must, as SumTotal president Kevin Oakes recently wrote in T+D (2004), “earn a seat at the table.” My goal is to help you get there and to be invited back again and again. It's probably a better job than the one you have now.

April 2004

More about Metrics

Posted by Jay Cross at April 19, 2004 06:35 PM | TrackBack

I don't know.

Posted by: flug berlin at July 9, 2004 01:42 PM

30 Poppy Lane
Berkeley, California 94708

1.510.528.3105 (office & cell)

Subscribe to this Blog

Enter your email address to subscribe. We vow never to share your information with anyone. No Spam.

Subscribe Unsubscribe

Reference Pages

Building Community
CSS, Semantic Mark-Up, and codes
First Principles
How People Learn
Knowledge Management
Learning Links
Learning Standards
Making It Work (Implementing)
Metrics & ROI
Social Software
String theory
The eLearning Museum
Visual Learning


Our Infrequent Newsletter
Sign up for our sporadic newsletter.

Entries by category...

Customer care
Emergent Learning
Just Jay
Recycled from Blogger
The Industry
Workflow-based eLearning


Internet Time Group

© 2004 Internet Time Group

Click for Berkeley, California Forecast
Berkeley, California

Recent entries

New Blog
Blogger Experience, Housekeeping, Something New
Loosely Coupled
Above all
Demographics is destiny
Are you setting the bar high enough?
Virtual Apps
Aerobic Learning
Work as Video Game
Oracle and Macromedia, Sitting in a Tree
The Blogosphere
ASTD Silicon Valley
Performance Support
Kingsbridge Conference Center
First Post by Email
Inactive Blog
RSS Feed for New Site
Comment Spam
Testing ... testing ... 1...2..3
IT Doesn't Matter - Learning Does.
All blogging is political
Mutlimedia Learning
Damn, damn, double damn
Nonverbal impact?
The New Religion
Wolf! Wolf! Wolf! Wolf! Wolf! Wolf!
Business Process Management (2)
Really Big
Business Process Management Conference
Don't Lose a Common Sense: LISTEN
It's only natural
Go with the flow
Time Out for the Fair
Informal get-together in SF this Wednesday
Repetition, reverb, and echoes
Who Knows?
Cognitive Mapping
Push vs pull
The Big Picture on ROI
Art Break
TDF Finale
New Community of Practice Forming
More TDF04
Training Directors Forum 2004
A Rare One-Liner
PlaNetwork LIVE 2
PlaNetwork LIVE
ASTD 2004 Leftovers
Worker Effectiveness Improvement, not KM
Upcoming Events
eLearning Effectiveness?
Jay's Talk at ASTD
Mintzberg & Cooperider
Lest ye forget
ASTD International Conference & Exposition 2004
Knowledge Tips
What is Workflow Learning?
ASTD msg 1 of n
Look out, it's Outlook
Collaboration at ASTD Next Week
Tell me a story
User indifference
The shortest presentation on metrics you will ever hear
Back to Blogger
Windows fixes
The Alchemy of Growth
Grab bag
Very loosely coupled
E-Learning from Practice to Profit
Robin Good kicks off Competitive Edge
China Bloggers
Sonoma Dreaming
Upcoming Events
Emergent Learning Forum: Simulations
The Best Things in Life Are Free
Metrics and Web Services
OpEd: ROI vs. Metrics
e-Merging e-Learning
Loosely Coupled
Search me