Category: Training Evaluation Plan
The mission of an evaluation documents
- What we going to do?
- For whom are we going to do it?
The mission itself defines what you are doing – how you’re going to do it is Evaluation Plan. An example of an evaluation mission is…
As the word “mission” implies, the mission statement clearly states the evaluation’s goal. With this goal in mind, all stakeholders have an understanding of the evaluation direction and purpose. A mission statement is one of the simplest and most basic necessities of a training evaluation, yet it is frequently overlooked by evaluators.
When you embark on creating a training evaluation plan one of the first steps is to determine the process or how the evaluation will be performed. Whatever evaluation method you elect to use (Predictive Evaluation, The Kirkpatrick Model, The Success Case Model), the process to implement needs to be defined and accepted. The process I use is:
The Plan phase is the starting point of the training evaluation. Planning an evaluation is specifying what and how you’re going to evaluate by reviewing the program, developing the project’s mission, creating the evaluation objectives & deliverables, developing the questions the evaluation will answer, recording assumptions, documenting boundaries, identifying stakeholders, developing the strategy, estimated costs & timelines, and performing risk assessment. From this a detailed evaluation project plan with milestones is created.
- Developing evaluation procedures involves taking the approved evaluation plan from phase 1 and drafting the techniques, procedures, and instruments necessary to implement the plan.
- This phase of the training evaluation of entails ( collect phase) gathering data to address the evaluation questions created in the evaluation plan.
- In the analyze phase the meaning of collected data are interpreted. This phase uses the collected data to determine the conclusions for which the data supports and the amount of support the data supplies for, or against, those conclusions. Quantitative & qualitative data analysis procedures are employed.
- The purpose of this phase ( report ) is to create the vehicles to communicate training evaluation information to stakeholders so that they can make use of the evaluation information for decision-making or purposes of accountability. This phase will generate an evaluation report and an executive dashboard detailing the results of the evaluation and the recommendations for the city to maximize the program’s impact.
As always, I look forward to you comments and ways to improve this approach.
Failure to do proper training evaluation planning causes poor efforts that usually include rework, the project being over budget, and delays. The Plan phase is the starting point of the training evaluation. Planning an evaluation is specifying what and how you’re going to evaluate by reviewing the BOC-1 program, developing the project’s mission, creating the evaluation objectives & deliverables, developing the questions the evaluation will answer, recording assumptions, documenting boundaries, identifying stakeholders, developing the strategy, estimated costs & timelines, and performing risk assessment. From this a detailed evaluation project plan with milestones is created.
I begin all my evaluations with with a detailed plan in which I include a steering committee to build it. The steps I use with the steering committee are:
- Determine the evaluation methodology
- Understand the course being evaluated
- Mission of the evaluation
- Objectives & deliverables
- Questions the evaluation will answer
- Assumptions
- Boundaries
- Stakeholders
- On-the-job behaviors
- Beliefs
- Strategies (data collection, data analysis, reporting)
- Timeline (project plan & key milestones)
Over the next few weeks I will post how to do each step. I hope you find them useful and encourage you to give me your thoughts.
Many times I have T&D professionals ask me the steps I take when evaluating programs. Here are the steps I follow:
But this is only the framework for conducting an evaluation, you should create an Evaluation Project Plan . This is a formal, approved document used to guide both project execution and project control. The primary uses of the Evaluation Project Plan are to document planning assumptions and decisions, facilitate communication among stakeholders, and document approved scope, cost, and schedule baselines.
Currently I am about to begin Level 3 Evaluation project for a client. Here is the project plan we just agreed to:
This gives you an idea of the time per task/milestone and the activities (tasks) for each step in the framework.
Evaluation Deliverables
Using the evaluation questions and other planning elements you have defined earlier, create a list of information that the evaluation needs to deliver in order to answer the evaluation questions. Specify when and how each item must be delivered. These are the elements of the evaluation. Add the deliverables to the evaluation plan, with an estimated delivery date. More accurate delivery dates can be established during the scheduling phase, which is next.
The Evaluation Schedule
Create a list of tasks that need to be carried out for each deliverable. For each task, identify the following:
- The amount of effort (hours or days) required to complete the task
- The resource that will carry out the task
Once you have established the amount of effort for each task, you can work out the effort required for each deliverable and determine an accurate delivery date. Update your deliverables section with the more accurate delivery dates. A common problem discovered at this point occurs when an evaluation has an imposed delivery deadline from the sponsor or steering committee that is not realistic based on your estimates. If you discover that this is the case, you must contact the sponsor/committee members immediately. The options you have in this situation are the following:
- Renegotiate the due dates
- Employ additional resources
- Reduce the scope of the evaluation
Congratulations! Having followed all the preceding steps, you should have a good evaluation plan. Remember to update your plan as the evaluation progresses and measure progress against the plan.
Previous posts in this series
Questions Evaluation will Answer
Elements in an Evaluation Plan
Coming up in this series of posts:
- Performing the evaluation
- Training evaluation and continuous improvement
- Training evaluation organizational readiness
As always, I would enjoy hearing your thoughts on this topic.
Using the evaluation questions and other planning elements you defined earlier (see previous blog posts), create a list of information that the evaluation needs to deliver in order to answer the evaluation questions. Specify when and how each item must be delivered. These are the elements of the PE (Predicting Training’s Value, Intention Evaluation, Adoption Evaluation, and/or Impact Evaluation).
Add the deliverables to the evaluation plan, with an estimated delivery date. More accurate delivery dates can be established during the scheduling phase, which is next.
In addition to the evaluation questions, other elements of the training evaluation plan arise from stakeholder interviews and your research into the course. Each element that you should consider as part of the plan is shown below.
Course to be evaluated: List the title and purpose for the course. Provide a history (business reason) for the course and what it is intended to achieve. For new courses (ones yet to be developed), these data can be found in the needs analysis and/or instructional design documents. For existing courses, use the current training materials.