The Kirkpatrick Model: Context, Critique & Conclusions
This is the last part of the Kirkpatrick Model series. Let’s get started, by going back to the beginning, not the beginning of the Model but the beginning of the context surrounding how a model like the Kirkpatrick Model arose in the first place.
You might remember from the first blog in this series we briefly discussed the background of the Kirkpatrick Model. In that subsection, we provided some details about Donald J. Kirkpatrick, the founding of his Model, and his partnership with his children. The immediate context, though, has its roots in WW II and the rise of a field termed operations research.
The roots of operations research arose during the 19th century, but the field really became operational (pun intended) during WW II. By using analysis, mathematics, and statistics, decision makers were able to optimize their choices. This applies to a range of fields such military operations (e.g., targeting) or transportation (e.g., queuing theory). This may remind you of the sub-field of game theory, which gained academic legitimacy with the 1944 publication of the book Theory of Games and Economic Behavior by John von Neumann and Oskar Morgenstern (think John Nash and the movie A Beautiful Mind). It’s not a far stretch to imagine that all of these concepts would be useful to executives in a range of fields, witness the birth of management sciences and decision sciences.
Donald W. Miller, a former professor of mine, taught a class on operations research at the Columbia University Business School. Like others, he was involved in the war effort using various quantitative techniques to maximize efficiency. His 1960 book Executive Decisions and Operations Research formalized the already existing trend of management science as a sub-field of operations research. Of course, the Kirkpatrick Model arose from this much larger intellectual and professional context.
Detailing all of the critiques of the Kirkpatrick Model is far beyond the scope of this blog. In fact, such work might constitute a lengthy article or a book in itself. The fact the Kirkpatrick Model is a point of reference for training professionals is illustrative. That Kirkpatrick did not get the “original” model “true” or “complete” is not surprising; few ideas emerge fully developed and nuanced in their original form. As colleagues developed competing models, Kirkpatrick continued to refine his original thoughts and, naturally, expanded his articulation of his model. A major area of critique, however, was the issue scientific accuracy.
Operations research (or management science), attempts to understand why and how a decision maker’s choices (inputs) impact a given outcome (outputs). Doing so requires an analysis of the proverbial “black box.” Some critiques argue that the Kirkpatrick Model was flawed; in this regard, they argue that Kirkpatrick’s model (his black box) was inaccurate. When confronted with such comments, Kirkpatrick refers his critics back to his original work, which stipulates that the four levels were not a model (a black box) but simply a framework to guide decisions.
Another critique of the Kirkpatrick “Model” deals with Return of Investment (ROI). ROI focuses on an extrapolated analysis, which compares resources expended for a business goal versus the realized value associated with its output. Was it worth it? While one assumes such a calculation was in Kirpatrick’s mind when crafting his “framework,” it was formally added as an element of the “true” and “complete” New World Kirkpatrick Model, Return of Expectations (ROE).
Again, for the best articulation of Kirkpatrick framework and principles go to their web site: www.kirkpatrickpartners.com and/or look at the following white papers:
“The Kirkpatrick Four Levels: A Fresh Look After 50 Years, 1959-2009,” Jim Kirkpatrick and Wendy Kirkpatrick, April 2009;
“An Introduction to the New World Kirkpatrick Model,” Jim Kirkpatrick and Wendy Kirkpatrick, March 2015.
This six-blog series covered the Kirkpatrick Model and the New World Kirkpatrick Model. Despite critiques to numerous to detail, if remains as the standard point of reference in any discussion regarding training analysis. Upon your review and reflection, it may overlook issues germane to your field. This, really, should not be a problem. The “model” or framework is not intended to be a definitive guide to the extent of serving as an oracle; rather, if lends analytical tools to assist administrators and executives in their business decisions.
Craig Lee Keller, Ph.D., Learning Strategist