Research shows that the vast majority of CEOs want learning and development investments to be linked to business impact data. Yet, many CLOs feel that calculating return on investment information is too time consuming and too costly. To bridge this disconnect, learning and development professionals must become more familiar with ROI methodologies and best practices.
The ROI Process Model is a useful framework which breaks return on investment work into four stages: evaluation planning, data collection, data analysis, and reporting. Before any ROI calculations are undertaken, CLOs need to ensure that the organization’s learning activities are linked to broader business goals.
The next step is to gather information and isolate the effects of training on business results. It is widely recognized that this is challenging, but is not impossible. Techniques like participant and manager estimates, as well as control groups and trend line analysis, have been used successfully by numerous organizations. Corporate learning professionals can demonstrate the value of their efforts to business leaders by integrating ROI principles into the analysis, design, and evaluation stages of their work.
Corporate Learning: The Need for ROI
Learning and development represents a significant expenditure in larger organizations. In some companies, the learning and development budget exceeds $500 million and may reach as much as $1 billion (Phillips, 2009). Given the size of these investments, it is not surprising that business leaders are demanding accountability and visibility into the value delivered.
A recent survey by the ROI Institute found that 96 percent of CEOs want learning and development to be connected to business impact data, but only eight percent actually have access to this type of information. In addition, three quarters of CEOs (74 percent) want return on investment (ROI) data related to learning and development, but only four percent are provided with this information (Phillips, 2010).
Unfortunately, many chief learning officers (CLOs) are influenced by incorrect perceptions about ROI:
ROI data for corporate learning isn’t necessary, because no one has ever asked for it. A reactionary attitude toward return on investment information can have troubling consequences. Executives may never ask for ROI data and the corporate learning budget may simply be cut without any discussion. Alternatively, if the request is made, there may not be adequate time to do the analysis properly.
It’s too time consuming. According to the ROI Institute, only about five percent of an organization’s learning initiatives need to be analyzed at the ROI level each year (Hartley, 2012). Time and effort should be spent calculating ROI for programs that are expensive or strategic.
It’s too costly. The ROI Institute has estimated that a comprehensive evaluation system in which every program is subject to some evaluation and a few are analyzed at the ROI level costs around three percent of an organization’s learning and development budget (Phillips, 2009).
The Kirkpatrick/Phillips Model breaks down learning evaluation into five levels: Level 1 – Participant Reaction, Level 2 – Level of Learning Achieved, Level 3 – Changes in Learning Behavior, Level 4 – Business Results Derived from Training, and Level 5 – Return on Investment from Training. A study by the American Society for Training and Development (ASTD) and the Institute for Corporate Productivity (i4cp) found that 92 percent of organizations measure learning programs to at least Level 1. However, Level 1 information is the least valuable. In contrast, only 18 percent of organizations evaluate learning programs at Level 5. Yet, nearly 60 percent of companies that engage in this level of measurement find that the data has high or very high value (Patel, 2010).
Approaches for Calculating Learning ROI
Before a CLO can calculate the ROI of corporate learning, he or she must ensure that the organization’s learning initiatives are linked to business goals. A good first step is to identify what matters most to the CEO and board of directors with regard to learning metrics. Consider how two large companies link corporate learning to business objectives:
At Ingersoll Rand, learning programs are co-designed with business leaders. First, the business goal is articulated. Then the necessary knowledge and behaviors are identified. Based on this information, training designers present business leaders with different learning options. A business sponsor selects the solution which provides the best value and business relevance for the company (Hartley, 2012).
The CLO at Agilent Technologies uses both operational and strategic metrics to evaluate learning initiatives. Operational metrics are quantitative in nature. Examples include training dollars per participant, total training dollars spent, and hours of training per employee. These metrics are collected to ensure that time and resources are spent well. Strategic metrics focus on the organizational impact of learning initiatives. Some strategic metrics are quantifiable, while others are not (Hartley, 2012).
To ensure that learning ROI is credible, it is important to isolate the effects of the training program on business results. This can be a major challenge for many organizations. ASTD and i4cp found that 52 percent of survey respondents felt that isolating how learning affects business results is a high or very high barrier (Patel, 2010).
A study by the ROI Institute identified several best practices for isolating the impact of learning on organizational results. The most frequently used technique was participant estimates, followed by manager estimates, control groups, and trend line analysis (Phillips, 2011). To illustrate the use of participant estimates, consider the scenario where a sales team takes a selling skills course and after six months, sales revenues have increased 15 percent. Although other factors were in play during the same period, such as promotions and commission increases, the sales team is the most credible group to articulate how the different factors contributed to the increase in sales.
When isolating the impact of training, multiple approaches may be equally credible. In this case, it is a good practice to use the method that generates the lowest ROI. This reasoning will resonate with the CEO and CFO (Phillips, 2011).
The ROI Process Model is a useful framework for determining the value delivered by corporate learning initiatives. As outlined in the table below, its steps can be mapped to the Kirkpatrick/Phillips Model which is familiar to many corporate learning professionals.
|ROI Process Model|
|Evaluation Planning||The corporate learning team develops and reviews the objectives of the program and connects the program to broader business objectives. The team develops evaluation plans and baseline data for the learning program.|
|Data Collection||Data collection during program satisfies Levels 1 and 2 in the Kirkpatrick/Phillips Model. Data collection after program satisfies Levels 3 and 4 in the Kirkpatrick/Phillips Model.|
|Data Analysis||The corporate learning team:
|Reporting||The corporate learning team creates an impact study.|
Source: “Moving from Evidence to Proof.” T+D, August 2011.
Generally speaking, the return on corporate learning can be determined most rapidly in project-specific environments. For example, a business team brings a project specific challenge to the learning and development group. The trainers create courses that are customized to address the issues. If the project is a success, the organization can measure direct cost savings as a result of the learning activities (McCrea, 2009).
Integrating ROI into Existing Learning & Development Models
For many years, learning and development professionals have used the ADDIE Model when creating training programs. This model has five stages: (1) Analyze the need, (2) Design the solution, (3) Develop the solution, (4) Implement the solution, and (5) Evaluate the solution. One of the major weaknesses of the ADDIE Model, however, is that evaluation is considered after implementation – this is too late (Phillips, 2010).
To address this problem, chief learning officers can measure ROI at two additional stages in the ADDIE Model.
Forecast. During the Analyze phase, it can be useful to apply return on investment principles and estimate whether a training initiative will have a positive outcome for the company. When developing sales training, for instance, the corporate learning team may want to forecast how many more deals will be closed per month, due the course. This can be translated into estimates of additional revenue for the organization. Comparing the expected benefits to the program costs will result in an estimated ROI of the training (Mattox, 2011).
Plan. During the Design phase, corporate learning professionals may identify several different training options that could meet the business’s needs. ROI techniques can be helpful to estimate the benefits of each based on their anticipated effectiveness, as well as the expected expense. The option with the most favorable ROI should be selected.
ROI methodologies can also be used during the Evaluate stage of the ADDIE Model. The ROI Process Model described above is a good way to quantify the value of a specific learning program. In addition, CLOs may want to gather ROI data for multiple training initiatives and compare outcomes for different courses. These findings will better demonstrate the value of Corporate Learning to business leaders throughout the organization.
EBSCO is a leading provider of learning content to corporations world-wide. The Leadership & Management Learning Center is a turnkey solution designed to sustain formal and informal learning programs and encourage contextual learning. The solution provides pre-populated, highly relevant content from industry experts on leadership and management best practices and benchmarks. The content is organized by competency so users can find the information they need quickly and easily. Content includes book summaries EBSCO’s Business Book Summaries, a collection of top leadership and management eBooks, journals and magazines from our Leadership & Management Source product, videos, and tests to ensure comprehension.
EBSCO provides the flexibility needed to implement learning content into exisiting learning programs. Customers also can use the EBSCOhost platform for informal learning. EBSCOhost supports key metrics to facilitate estimating learning ROI. These include summary usage statistics (for demonstrating the utility of EBSCO learning content,) and comprehension tests (for demonstrating the amount of knowledge transfer from EBSCO content).
Hartley, D. (2012). Relevance trumps ROI. Chief Learning Officer, 11(2), 36-39. Retrieved August 10, 2012 from Business Source Corporate Plus.
Hill, T. (2009). Measure smart: Trade ROI for IOB. Chief Learning Officer, 8(8), 28-31. Retrieved August 10, 2012 from Business Source Corporate Plus.
Lovell, K. (2012). Evaluating learning. Training Journal, 53-56. Retrieved August 10, 2012 from Business Source Corporate Plus.
Mattox, J. (2011). ROI: The report of my death is an exaggeration. T+D, 65(8), 30-32. Retrieved August 10, 2012 from Business Source Corporate Plus.
McCrea, B. (2009). The ROI from executive education. Supply Chain Management Review, S1-S5. Retrieved August 10, 2012 from Business Source Corporate Plus.
Patel, L. (2010). Overcoming barriers and valuing evaluation. T+D, 64(2), 62-63. Retrieved August 10, 2012 from Business Source Corporate Plus.
Phillips, J., & Phillips, P. (2010). Confronting CEO expectations about the value of learning. T+D 64(1), 52-57. Retrieved August 10, 2012 from Business Source Corporate Plus.
Phillips, J., & Phillips, P. (2011). Moving from evidence to proof. T+D, 65(8), 34-39. Retrieved August 10, 2012 from Business Source Corporate Plus.
Phillips, J., & Phillips, P. (2009). The real reasons we don’t evaluate. Chief Learning Officer, 8(6), 18-23. Retrieved August 10, 2012 from Business Source Corporate Plus.
Prager, H., & Vece, S. (2009). Simplified ROI: Measuring what matters most. Chief Learning Officer, 8(11), 28-31. Retrieved August 10, 2012 from Business Source Corporate Plus.