Category: Impact Evaluation
An Impact Dashboard summarizes the Impact found. A typical Impact Dashboard is made up of four pages/sections: (1) payback, results, and key findings; (2) impact by Adoptive Behavior; (3) impact examples and recommendations; and (4) the Impact Profiles.
Sample Impact Dashboard – page 1
Sample Impact Dashboard – page 2
Sample Impact Dashboard – page 3
A Different Impact Dashboard
Impact Evaluation Summary
An Impact Evaluation addresses the following questions:
- What business results can be traced to the goal adoption of participants?
- Are results as predicted to the business?
- What is the typical profile of a participant who has achieved results?
- What additional business results (value) could be achieved if graduates who have little/no adoption were to adopt their goals?
The reason to train should be to improve the company’s bottom line. By measuring the effect on the company profits from participant intentions, adoption, and impact and comparing that with the predicted value, a company can determine its return on investment and can maximize organizational and business results.
Previous Blogs in the Impact Evaluation Series
Collect Detailed Impact Data from Successfully Adopted Participants
Interview Successful and Unsuccessful Participants (part 1 of 2)
Interview Successful and Unsuccessful Participants (part 2 of 2)
In Predictive Evaluation (PE), I share my evaluation findings with stakeholders using the dashboard concept. The dashboard is a report to senior management providing an at-a-glance perspective on the current status of a course in the context of predetermined (predicted) metrics for that program (Intention, Adoption, or Impact). PE dashboards are simple, short, and do not require interpretation. Click here for a sample of these types of
Predictive Evalution
dashboards.
Recently, some of my clients have needed year-end reports that share evaluation findings ( Intention , Adoption , and Impact ) in a single dashboard. I thought that sharing this new year-end dashboard with you was an excellent topic for my final blog of 2011. When designing the new dashboard I set the following requirements:
During
Impact Evaluations
, we often interview to participants to determine their results realized since completing training. Here are some tips for conducting an effective Impact Evaluation interview.
In Predictive Evaluation when we are conducting an Impact Evaluation, we interview participants to collect results data.
High-Impact interview questions should be focused on such issues as the following:
- Validate what participants have actually done. What actions have they undertaken? How did training contribute toward the results?
- What knowledge and skills from training did they use? What tools/techniques learned in training have they employed? What knowledge or tools did participants have to learn on their own to produce the results?
- What in the environment has enabled or inhibited them as they produced results?
- What are the tangible business results? Gather specific examples. How much of the business result can be directly attributed to training? Are the results sustainable?
- Will they continue to use these new actions and how do they see them being repeated in the future?
Use a semi-structured interview process with probing questions to gather additional data as needed. A semi-structured interview directs the interview conversations. Some of the interview questions may or may not be posed according to the conversation between the evaluator and the participant. The evaluator may pose emerging questions (not listed in the interview protocol) as appropriate according to the interview conversation. Each interview is anticipated to last no more than thirty minutes.