Category: Uncategorized
Just had a new article published with one of my best clients: Michael Yarter in Chief Learning Officer titled: How to Set Up an Internal Training Evaluation Department . Michael and I have both set up internal evaluation departments and collaborated on what we call the “imperatives” for doing this properly to write this article.
I hope you find it useful and would enjoy hearing your comments and experiences on how you have done this.
So honored that training magazine just published my article on Getting Managers Engaged in Employee Training . This is one element of my Learning to Performance (L2P) approach. L2P is a complete training approach that combines learning, leadership and change management competencies to produce documented, sustainable results and value.
I hope you enjoy the article and I would love to hear what you have done in this area.
In addition to the evaluation questions , other elements of the evaluation plan arise from stakeholder interviews and your research into the course. Each element that you should consider as part of the plan is shown below.
Course to be evaluated. List the title and purpose for the course. Provide a history evaluated (business reason) for the course and what it is intended to achieve. For new courses (ones yet to be developed), these data can be found in the needs analysis and/or instructional design documents. For existing courses,use the current training materials.
Purpose of the evaluation. Documents why the evaluation is being undertaken and how the information will be used (comes from stakeholder requirements). Example: Our approach is based on moving away from the typical measurement of activity or level-based measures in favor of focusing on value-driven continuous improvement efforts.This ensures that the training investment creates value and produces the behaviors and results that lead to the desired business outcomes.
Questions the evaluation will answer. These are the basic building blocks for the training evaluation. They are the questions that key stakeholders would like to see answered. Examples: Are participant intentions and beliefs upon program completion aligned with desired values? Are participants using (adopting) the skills on the job? Is the training program contributing to achievement of the business outcomes? If yes, to what degree? If no, why not?
Key Stakeholders. These are the individuals who have a vested interest in the evaluation information.
Once this is completed, it’s time to move on and look at the evaluation deliverables (next week’s post).
Previous posts in this series:
Questions Evaluation will Answer
Coming up in this series of posts:
- Evaluation deliverables
- Evaluation schedule
- Performing the evaluation
- Training evaluation and continuous improvement
- Training evaluation organizational readiness
As always, I would enjoy hearing your thoughts on this topic.
Over the past few years I have been pondering the question “Is training an outcome driven business or an activity driven function?” Think about it. Too many times we design training that links content to intended business results, deliver the course resulting in participant skill/knowledge change, but then just report the activity.
What I continually see in my experience are activity measures being reported by training functions versus the key impact to business results. For example, those being:
- Number of employees trained
- Hours trained
- Cost to train versus budget
- Level 1 end-of-course reaction results (one could argue that these data begin to report outcomes – those being participants’ reaction or satisfaction with the training experience)
I believe that these metrics are necessary and essential but they are activity metrics not outcome measures which are even more essential. Outcome measures should state the value that training has brought to the business . In other words: impact. So what are some outcome driven business metrics? There are many and they need to be chosen so that it is within the context of your company and its typical reporting measures. Here is the approach I use:
If you think of a training program as having a product life cycle (the stages through which a course bypasses) consider using 5 stages: 1) Forecast, 2) Design & Development, 3) Delivery, 4) Transfer, and 5) Impact.
Forecast. This helps management in its attempts to cope with the uncertainty of the future, relying mainly on data from the past and present and analysis of trends. Forecasting starts with certain assumptions based on the management’s experience, knowledge, and judgment. Elements I use in a training forecast are:
- Budget for all stages
- Design & development cycle time
- Number of employees trained
- Predictive Evaluation Impact Matrix ( Intention , Adoption , Impact )
- Predictive Evaluation ROI
Forecasting usually is coupled with a needs analysis in which the learning objectives are provided.
Delivery. The activity metrics used in delivery are typically:
- Expense versus budget
- Number of employees trained
- Training hours delivered
During delivery, outcome measures can be used. The measures I use are:
- End of course reaction
- Predictive Evaluation Intention Score
- Predictive Evaluation Belief Score
Transfer. The transfer outcome measure is the rate and degree that participants have applied training skills on the job. The transfer outcome measure I use is the Predictive Evaluation Adoption Rate.
Impact. Successful transfer should lead to increased employee and organizational performance which results in Impact – tangible business results from training. The impact outcome measure I use is the Predictive Evaluation ROI.
Thank you for reading and as always, I invite your comments and thoughts.
This last weekend I was in Baltimore, MD at a Berrett- Koehler authors retreat. I participated in a breakout session on Improvisational Writing where we all did free writing for a set time limit on a phrase. So, today, I move away from writing about training evaluation and share with you two of attempts. Good or bad here they are. Thanks to Devora, Jennifer, John, and Mark for being there with me.
Somewhere a Saxophone Wailed
Sam walked up to the home late at night after being aroused from bed by a call from the precinct.
“Looks like we got another one,” the dispatcher said.
“Same MO?”
“Yep, two gunshot wounds.”
“And the other?”, he queried.
A paused and then finally. “Weird but yes, two shots and jazz music playing a CD next to the body.”
Sam entered the home, saw the blood splatter on the wall, and heard the music playing in the background. Just like the other five times; the saxophone killer had struck again.