Fresh Look at ROI|
return-on-investment meets the information age.
Ask me, I'll
tell you: Return-on-investment isn't what it used to be.
financial measure, developed by DuPont and once credited with making
General Motors manageable, hasn't kept pace with the times. The
R is no longer the famous bottom line and the I is
more likely a subscription fee than a one-time payment.
most training decisions were incremental. Training sponsors had most
of the infrastructure required: an empty room, staff, flipcharts,
markers, perhaps some personal computers. Business unit managers
could evaluate the cost-effectiveness of one-shot training courses
by assessing cost and effect within their own business units.
E-learning changes this, though.
E-learning is a
continuous process, not a one-shot deal. It is most often an
enterprisewide initiative, beyond the bounds of any individual
business unit. And investing in e-learning is often a strategic
imperative--the entry ticket to an e-business environment.
for different folks
Where you stand
on ROI depends on where you sit. Different levels of management make
different sorts of decisions, so it's appropriate that they use
different measures of ROI.
||Close skills gap
specific training |
goals, business metrics|
the best alternative
metrics, business case
||Enterprise, e-learning infrastructure|
competitive advantage, transformation
case, shareholder value |
All these goals
and measures are valid--in fact, they complement one another. The
more each managerial group understands the others, the better the
odds they'll make sound decisions.
domain of business decision making is at a crossroads; the old
yardsticks no longer apply. Why measure incremental improvements
when you're seeking the Holy Grail? Rational decision makers look
beyond an ROI that reduces everything to the lowest common
denominator. Traditional financial analysis works in stable times
but falls apart when things go off-scale.
Bearing that in
mind, let's examine ROI from four different perspectives: training
manager, business unit manager, corporate staff, and executive
your training manager is a chief learning officer who has the ear of
top management, or a wave of re-engineering swept your corporate
staff out the door. When these mismatches pop up, substitute the
person in your organization who has these interests, positions, and
ways of doing things.
The training manager
Don't step in
Five or six years ago, I witnessed a multimedia course demonstration
designed to help bank officers spruce up their table manners. When a
banker clicked on sushi, a voice intoned "sooo-she." Then a
Japanese flag appeared, followed by words explaining that sushi is a
popular Japanese dish made of fish and rice. I chuckled and thought
that's going to be one surprised banker when the raw fish arrives.
The sushi story
came to mind last fall during a series of breakout sessions at the
TechLearn and Online Learning conferences. But this time, the topic
drove home the message relentlessly to dozens of groups: If you want
to sell a big project internally, you've got to talk ROI. It's the
language senior managers understand, and it's how they separate
worthy projects from losers. Being fluent in ROI talk enables you to
position a learning project as an investment rather than a cost,
they said. It's the secret handshake that gets you into the inner
circle of those who control budget dollars.
reality-check time. Talking the ROI talk won't enable you to pass
yourself off as an astute businessperson. You have the same chance
of passing for French with a beret and Berlitz phrase book. A little
knowledge can be dangerous. Making a significant business decision
entails a wide range of factors and involves intricate tradeoffs.
- Risks must be
weighed against rewards.
aims need to be sorted from the long.
must align with strategic initiatives.
resources call for shrewd horse-trading.
training unit sells training for a fee--generating its own
revenues--the returns on training investment come from satisfying
the needs of business unit managers. Tying training results to
business results is more useful than coming up with pseudo-ROI
The only valid
training ROI is business ROI.
trump numbers. International Data Corporation studied the buying
behavior of corporate and IT training managers and concluded that,
"ROI will no longer be measured in savings or reduced cost of
training." Instead, attention will be directed to "measurable
changes to business metrics resulting from training investments.
Those benefits will only emerge if vendors increase their focus on
high-quality instructional design and engaging learning
exercises are necessary to show you've done your homework; the
numbers are your ticket to the adults' table. A decision maker's
final choices, however, are mostly visceral--based on how the likely
outcome of each alternative makes him or her feel. Feelings win out
because the assumptions used to create the numbers can always be
challenged. Projects that evoke the best feelings make the cut.
To sell a
project upward, you've got to make the upside come alive--paint a
picture, tell a story. And make no mistake about it, upselling is
what you need to do.
The business unit
managers own the problems that training solves. They're generally
pragmatic, and their overriding interest is getting a job done.
Soon. Until you know what they're trying to accomplish, you can't
talk with them about potential results.
unit manager is usually training's primary sponsor. When you're
working with the right client--the decision maker who understands
the end goal and controls the environment in which the problem
occurs--measuring results can be simple. Start with business
problems and work backwards. The most important step in measuring
performance is pinning down the business manager's answer to the
classic question: What's in it for me?
Don't skip this
step. Without it, meaningful tracking is impossible. First gain
agreement on the problem and the value of solving it. Then outline
how you propose to solve it. Establish a baseline measure of current
performance, and clearly indicate how performance will be tracked
Proof is a
figment. Brilliant people assure me that it's impossible to isolate
the impact of training. You can never tell whether some concurrent
event has contaminated the results and negated their value as
scientific evidence of training's impact, they say.
To which I
reply, "Baloney!" (Not an exact quote.) Business decisions are made
with less-than-perfect information; it comes with the territory.
Management is not conducting a science class--it's looking for
So the question
isn't, "How do we prove beyond a shadow of a doubt that a given
training program produced a given result?" Rather, it's "What will
our sponsor accept as persuasive evidence that the program produced
the result?" Working with strong probabilities, make the case
logically, linking learning to business results. Establish a causal
link between a particular skill deficiency and a particular business
outcome. If the manager buys into this logic and the way you will
measure requests after training, that will be the proof you need.
How to track
business unit results. The process of tracking
learning results starts before any learning takes place. It begins
with partnering between the training manager and the business unit
manager who owns the problem. So, it's important to agree on the
value of solving the problem.
In most cases,
you gather information through interviews that focus on the work
process, not training. And they should always return to the mother
of all business questions: What difference would this make?
examination of the problem will pinpoint the gap between the results
the manager wants and the results he or she will actually get. Then,
determine what major skill gaps and learning deficiencies might hold
the expected dollar value gained by eliminating the deficiency and
make tangible projections from those outcomes. Make sure to get
signoff on the expected outcomes, how they'll be measured, and
performance criteria. This keeps discussions focused and agreements
throughout the process, you're helping managers answer questions
about why skills matter and what good performance looks like. You're
focusing sustained attention on solving problems and adding value,
and you're identifying tangible values for each skill to be taught.
As a result,
you're forging a partnership with the business unit client based on
his or her core concern: performance.
has been completed, assess the results according to measurements set
up with the manager. Extrapolate behavior changes into measurable
business. There's no room for vagueness--and no backing away from
visible quantitative evidence. Further interviewing and a review of
business results may be useful.
your findings and a simple cost-benefit analysis to the business
manager or training sponsor.
The corporate staff
has the resources to do all the good things it might; senior
executives must choose where to place the company's bets. A major
role of corporate staff is helping execs make sound choices.
Thorough analysis reduces the risk of choosing the wrong training
alternative. The bigger the project, the more analysis is justified.
analysis, an effective staff distills a complex business alternative
into a three- or four-page business case. Solution Matrix's Marty
Schmidt develops business cases for corporations, and he puts it
this way: "A business case is a tool that supports planning and
decision making, including decisions about whether to buy, which
vendor to choose, and when to implement. It is generally designed to
answer the question: 'What are the likely financial and other
business consequences if we take this or that action (or
The Solution Matrix Website
shares tips on how to assemble an effective business case. Here are
a few words of advice.
- Explain where
the data comes from to maintain credibility. Describe any
assumptions the reader wouldn't make automatically.
- Break out
"hard" benefits from "soft," because some readers will value them
differently. The difference? It's easy to assign a dollar figure
to a hard benefit (for example, reduce travel expense by $180,000)
but difficult for a soft benefit (free up professional time).
- Develop a
statement of the project's net cash flow.
every benefit and cost possible.
- If an item
simply can't be quantified ("Morale will improve," for example),
include it in a nonfinancial analysis, and rank it among the
financial impacts to show its relative importance.
- Include a
sensitivity analysis to show what happens when assumptions change
and a risk analysis to show that likelihood.
- Assume that
the numbers don't tell the whole story. Make the case that the
project boosts sales, improves service, speeds things up, and so
forth, then relate those benefits to business objectives.
business case uses financial metrics (ROI, internal rate of return,
discounted cash flow, and so forth) and numbers are precise, it's
tempting to imbue financial metrics with too much importance.
Numbers are another perspective, not another reality.
Beware of bad
Present-day accounting is an anachronism. It worked well when people
could go out to the warehouse and count assets. In the information
age, it's an inappropriate yardstick because most of the assets
drive home every night.
In a nutshell,
traditional accounting recognizes physical entities; intangibles are
valued at zero. Vast areas of human productivity--ideas, abilities,
experience, insight, esprit de corps, motivation--lie outside its
vision field. It doesn't recognize that people become more valuable
Your gut may
tell you that you'll be repaid in the future for investing in people
today, but where training is concerned, the only metric taken into
consideration is cost. Accounting has no measuring stick to
distinguish a good idea from a bad one. Excellent training hits the
books at the same value as bad.
is there? Weigh subjective factors as well as objective ones. Keep
two sets of books, one for investors and regulators, the other to
track the intangibles.
Robert S. Kaplan
and David P. Norton, co-authors of The Balanced Scorecard,
devised a means of evaluation developed to help make up for the
insufficiencies of financial accounting. In addition to finances,
the balanced scorecard looks at changes in customers, processes, and
employees. The method was designed to look backward--to evaluate in
hindsight--but there's no reason not to use it to project into the
future as a decision-making tool.
Kaplan and Norton, managers have traditionally attempted to improve
performance by making operating and investment decisions to develop
new and better products, to increase sales, and to reduce operating
costs. "Over time, however," they write, "it probably occurred to
some managers that during difficult times, when sales were
decreasing and operating costs were increasing, profits could be
earned not just by selling more or producing for less, but by
engaging in a variety of nonproductive and typically
is committed to implementing strategic changes to transform the
enterprise and increase shareholder value. But the bottom
line--profits left over after subtracting costs from revenues--is no
longer the be-all and end-all of business results.
typically focus on two things: strategy and outfoxing the
competition. They realize that competing successfully requires teams
of inspired employees who are mentally equipped to make sound
decisions on the fly; able to execute good ideas in a snap; and are
proactive when it comes to taking initiatives and bringing
innovation. The overall goal: an environment where people learn
faster and better than the competition.
takes more than a lavish investment in training, though. Time is
frequently more important than money. According to Klaus Schwab,
founder and president of the World Economic Forum: "We are moving
from a world in which the big eat the small to a world in which the
fast eat the slow."
Let's look at
how senior decisions are made. The staff has shopped various
projects, gathered the figures, done due diligence on suppliers, run
the numbers, assessed the impact of changes in the marketplace, and
prepared terse summaries for each scenario. A couple of projects are
no-brainers. These are so integral to the organization's mission
that giving a go-ahead is a formality.
enter new territory--e-learning for example--warrant more detailed
has shown that most senior executives have more faith in gut
feeling` than in numbers. The numbers are input, but the decision is
broader than that. Results from an Information Week survey
reveal that "More companies are justifying their e-business ventures
not in terms of ROI but in terms of strategic goals. Creating or
maintaining a competitive edge was cited most often as the reason
for deploying an application."
people--me included--the traditional concept of training ROI is
obsolete. Astute training managers employ business metrics, not
evaluation levels, I believe. Business unit managers value time more
than ROI. Major decisions are based on descriptive business cases,
not pro forma budgets. Senior executives tend to be more interested
in the top line (dramatic growth from new markets and innovation)
than the bottom line (the accounting fiction of profits).
The Net has
|Learning Circuits welcomes your feedback.
Jay Cross is CEO of Internet Time Group, a
California-based think tank and learning consultancy. Contact him at
note: This article was adapted from Cross's white paper, "A Fresh
Look at Return on Investment." Additional resources are found online
at the Internet Time