We are beginning to close up our discussion on the Kirkpatrick Model with a description of its last level, Level 4, which deals with the issue of Results.
LEVEL 4: RESULTS
Results . . . in short, yes, this is what administrators ultimately look at in the context of their training plans. Be it a commercial business or a not-for-profit, mindful organizations are extremely careful with costs that some might view as discretionary, such as training. For training to be of value, it ultimately needs to be translated into Results.
The Kirkpatrick Model characterizes Results as: “the degree to which targeted outcomes occur as a result of the training and support and accountability package.”
The KNWM adds another dimension to Level 4: Leading Indicators. This addition focuses on “short-term observations and measurements suggesting that critical behaviors are on track to create a positive impact on desired results” (www.kirkpatrickpartners.com).
The last level should be analyzed and structured even before Level 1 and even before the training begins. Why? First, administrators find it difficult to determine useful metrics for measuring employee behavior. Attempting to create metrics after the completion of the training is problematic, because doing so can lead to accepting poor measures of outcomes or just accepting a “general sense” of the outcome without looking at how the training actually impacted the bottom line. Let’s look at an example of possible consequences:
Imagine that training costs $10,000; imagine that training only increased production value by $1,000 per year; and, imagine that there is a complete turnover of employees every eight years. Such measurements confirm that the training decreased overall organization income by at least $2,000.
Second, administrators need to be able to discuss not only the training with employees but also the means by which they plan on measuring its value. Educational Technologies confirm that consulting with employees makes the collection of data for the metric easier; problems identified with the collection process can be fed back into the training program to modify future assessments (www.educationaltechnology.net).
Educational Technologies also suggests training value can be determined by introducing a “control group,” as one might in a formal scientific experiment. Creating a control group might seem to be discriminatory toward those not included in the training. However, such need not be depending upon the structure and timing of the training. For example, if the training takes place over rotating, consecutive phases that last, say, over six months, then it would be possible to assess performance metrics of the first group versus that of the last group still awaiting training.
As noted at the onset, the Kirkpatrick Model changed its conceptualization from a hierarchical pyramid toward links in a chain. The notion of a chain connotes an interconnected process, but the Kirkpatrick Partners also use the notion of chain to develop the means of determining Results: a Chain of Evidence.
Those at Kirkpatrick Partners argue that the chain model needs to be followed while being mindful of five different principles:
- The end is the beginning
- Return on Expectations (ROE) is the ultimate indicator of value
- Business partnership is essential to bring about positive ROE
- Value must be created before it can be demonstrated, and
- A compelling chain of evidence demonstrates your bottom line value.
Ultimately, these principles have led to what the Kirkpatrick Partners term as the “true” model or “complete model,” the Kirkpatrick Business Partnership Model as depicted below (“The Kirkpatrick Four Levels: A Fresh Look After 50 Years, 1959-2009,” Jim Kirkpatrick and Wendy Kirkpatrick).
Next week, we’ll describe the Kirkpatrick Principles in detail and, in a final blog, discuss the critiques of the Kirkpatrick Model while placing it in the context of other models.
Craig Lee Keller, Ph.D., JAG Learning Strategist