The Kirkpatrick Model: Principles

O.K.! With this blog, we’re finishing our description of the Kirkpatrick Model by detailing its Principles. Before that part, however, we really need to recap the previous blogs in this series. Why? It’s so easy to forget or simply get trapped by details. In short, we need to be able to see the forest for the trees (with the Kirkpatrick Business Partnership Model [KBPM] being the forest). So, quickly . . .

(click to enlarge)

The KPBM obviously has many similarities with the levels, though the order seems to have been reversed. Why is that? Let’s look at the first Kirkpatrick principle.

 

KIRKPATRICK PRINCIPLES

Let’s remember that the Kirkpatrick Partners argue that the chain model is the best way to appreciate the interrelated nature of assessing training programs. And, of course, the reason for the training program is because of a business need that has been identified.

  1. The end is the beginning. This principle reminds us that any training program—really any business decision—should be directly linked to a business need that was established at the onset.
  2. Inventor Don Kirkpatrick realized that assessing a training program necessitates understanding the organizational framework. This conditions data collection, surveying learning, and monitoring subsequent work behavior, in other words a chain of understanding and evidence. Administrators will be forced to rely on anecdotal comments and impressions if they don’t keep the end (the business need) in mind.
  3. Return on Expectations (ROE) is the ultimate indicator of value. In short, administrators need to understand that the money spent on training and assessment should translate into a positive organizational net gain. This part is quantitative, but it’s not necessarily simple. Program managers need to be able to envision what “success” would look like to them. In so doing, those designing training will be understanding business desires/needs while helping administrators and managers refine their business goals and expectations. Business partnership is essential to bring about positive ROE. When the Kirkpatrick Partners speak of a business partnership, they are redirecting the focus of training from the traditional focus of course content and employee knowledge. First, and yes, course content is extremely important, however, it’s not an end in itself; the end is the ROE. Bringing about a positive ROE will be impossible if employees fail to apply their learning, especially if it is forgotten after a period. That’s why the business partnership is key.
  4. he partnership is amongst employees, managers, and the administrator. Managers must be able to coach and encourage employees, and the administrator and managers must be able to create and offer incentives for success. This is one of the reasons why it’s important to be able to visualize success during the phase of training design.
  5. Value must be created before it can be demonstrated. In the aforementioned Kirkpatrick “A Fresh Look,” they call upon an industry study that identifies the sources of training failure. The largest area of failure by far was the application of the training in the work environment (70%). Principle 4 is a direct correlate of Principle 3. What do Principles 3 and 4 mean when taken together? Simply that training professions need to radically adjust their understanding of role. Instead of solely being the traditional, knowledgeable, empathic instructor, they need to guide organizations (administrators, managers, and employees) in a plan that includes operational execution and oversight. A compelling chain of evidence demonstrates your bottom line value. Principle 5 brings us back to the beginning, being able to demonstrate the ROE for the specified business need. The sequential nature of the levels and principles is based on the requirement to document value through the associated causation of the training and its follow-up. With this principle, the results are related to the business need, and organizations can begin the process of refining goals and modifying training practices.

Next week, we’ll finish up the series by comparing the KPBM with other models and placing this in the context of the modern business environment.

Craig Lee Keller, Ph.D., Learning Strategist

The Kirkpatrick Model: Context, Critique & Conclusions

This is the last part of the Kirkpatrick Model series. Let’s get started, by going back to the beginning, not the beginning of the Model but the beginning of the context surrounding how a model like the Kirkpatrick Model arose in the first place.

 

CONTEXT

You might remember from the first blog in this series we briefly discussed the background of the Kirkpatrick Model. In that subsection, we provided some details about Donald J. Kirkpatrick, the founding of his Model, and his partnership with his children. The immediate context, though, has its roots in WW II and the rise of a field termed operations research.

The roots of operations research arose during the 19th century, but the field really became operational (pun intended) during WW II. By using analysis, mathematics, and statistics, decision makers were able to optimize their choices. This applies to a range of fields such military operations (e.g., targeting) or transportation (e.g., queuing theory). This may remind you of the sub-field of game theory, which gained academic legitimacy with the 1944 publication of the book Theory of Games and Economic Behavior by John von Neumann and Oskar Morgenstern (think John Nash and the movie A Beautiful Mind). It’s not a far stretch to imagine that all of these concepts would be useful to executives in a range of fields, witness the birth of management sciences and decision sciences.

Donald W. Miller, a former professor of mine, taught a class on operations research at the Columbia University Business School. Like others, he was involved in the war effort using various quantitative techniques to maximize efficiency. His 1960 book Executive Decisions and Operations Research formalized the already existing trend of management science as a sub-field of operations research. Of course, the Kirkpatrick Model arose from this much larger intellectual and professional context.

 

CRITIQUE

Detailing all of the critiques of the Kirkpatrick Model is far beyond the scope of this blog. In fact, such work might constitute a lengthy article or a book in itself. The fact the Kirkpatrick Model is a point of reference for training professionals is illustrative. That Kirkpatrick did not get the “original” model “true” or “complete” is not surprising; few ideas emerge fully developed and nuanced in their original form. As colleagues developed competing models, Kirkpatrick continued to refine his original thoughts and, naturally, expanded his articulation of his model. A major area of critique, however, was the issue scientific accuracy.

Operations research (or management science), attempts to understand why and how a decision maker’s choices (inputs) impact a given outcome (outputs). Doing so requires an analysis of the proverbial “black box.” Some critiques argue that the Kirkpatrick Model was flawed; in this regard, they argue that Kirkpatrick’s model (his black box) was inaccurate. When confronted with such comments, Kirkpatrick refers his critics back to his original work, which stipulates that the four levels were not a model (a black box) but simply a framework to guide decisions.

Another critique of the Kirkpatrick “Model” deals with Return of Investment (ROI). ROI focuses on an extrapolated analysis, which compares resources expended for a business goal versus the realized value associated with its output. Was it worth it? While one assumes such a calculation was in Kirpatrick’s mind when crafting his “framework,” it was formally added as an element of the “true” and “complete” New World Kirkpatrick Model, Return of Expectations (ROE).

Again, for the best articulation of Kirkpatrick framework and principles go to their web site: www.kirkpatrickpartners.com and/or look at the following white papers:

“The Kirkpatrick Four Levels: A Fresh Look After 50 Years, 1959-2009,” Jim Kirkpatrick and Wendy Kirkpatrick, April 2009;

“An Introduction to the New World Kirkpatrick Model,” Jim Kirkpatrick and Wendy Kirkpatrick, March 2015.

 

CONCLUSIONS

This six-blog series covered the Kirkpatrick Model and the New World Kirkpatrick Model. Despite critiques to numerous to detail, if remains as the standard point of reference in any discussion regarding training analysis. Upon your review and reflection, it may overlook issues germane to your field. This, really, should not be a problem. The “model” or framework is not intended to be a definitive guide to the extent of serving as an oracle; rather, if lends analytical tools to assist administrators and executives in their business decisions.

 

Craig Lee Keller, Ph.D., Learning Strategist