Please join me for the first webinar from my webinar series Making the Most of Corporate Training Dollars: Predicting Training’s ROI.  Thursday, June 13, 2013  2:00-3:00 PM EST. Do you struggle to define training’s success? Are you fighting to justify the training’s value within your organization? Does your organization view training as an expense versus an investment with predicted return? Do you need a method of predicting (forecasting) the training’s value to help decide whether to train? Are your current evaluation efforts always "after the fact"? Do you want to measure success using leading indicators that drive continuous improvement?  WHAT YOU WILL LEARN In this complimentary webinar, training and evaluation I will spotlight Predictive Evaluation.  Using my innovative new Predictive Evaluation (PE) Model, trainers and business leaders can now successfully predict training’s ROI allowing them to make smarter, more strategic training and evaluation investments. Webinar attendees will learn how Predictive Evaluation enables you to effectively and accurately forecast training’s ROI to your company, measure against these predictions, establish indicators to track your progress (and make mid-course corrections if needed) and report the results in a language that business executives respond to and understand. This approach can be used for any sort of training program, in any setting, whether planned, newly implemented, or long-established.  Predictive Evaluation is collaborative, as supervisors and employees work together to establish standards for success each step of the way. The process helps guarantee that training results will be relevant to the business and gives all participants a sense of ownership in the process.  

Please join me on Thursday, June 13th at 2:00 PM Eastern for a complimentary webinar Predicting Training’s ROI Do you struggle to define training’s success? Are you fighting to justify the training’s value within your organization? Does your organization view training as an expense versus an investment with predicted return? Do you need a method of predicting (forecasting) the training’s value to help decide whether to train? Are your current evaluation efforts always "after the fact"? Do you want to measure success using leading indicators that drive continuous improvement? In this complimentary webinar, training and evaluation expert Dave Basarab will spotlight Predictive Evaluation. Using Dave’s innovative new Predictive Evaluation (PE) Model, trainers and business leaders can now successfully predict training’s ROI allowing them to make smarter, more strategic training and evaluation investments.  Dave, who (literally) wrote the book on Predictive Evaluation, regularly helps his clients maximize their training ROI by as much as 200 to 300%. In his webinar, he will explain how to accomplish this. Webinar attendees will learn how Predictive Evaluation enables you to effectively and accurately forecast training’s ROI to your company, measure against these predictions, establish indicators to track your progress (and make mid-course corrections if needed) and report the results in a language that business executives respond to and understand. This approach can be used for any sort of training program, in any setting, whether planned, newly implemented, or long-established. Predictive Evaluation is collaborative, as supervisors and employees work together to establish standards for success each step of the way. The process helps guarantee that training results will be relevant to the business and gives all participants a sense of ownership in the process. Space is limited.  Reserve your Webinar seat now at: https://www1.gotomeeting.com/register/478152048 Title: Predicting Training’s ROI Date: Thursday, June 13, 2013 Time: 2:00 PM - 3:00 PM EDT After registering you will receive a confirmation email containing information about joining the Webinar.

Please join me on Thursday, July 11, 2013 2:00-3:00 PM EST for this complimentary webinar on my Learning to Performance model. When companies diagnose performance gaps, training is often the solution. Most organizations - and trainers - rely on isolated training events, but Dave Basarab now offers his innovative Learning to Performance system, a complete training approach that combines learning, leadership and change management competencies to produce documented, sustainable results and value. Dave’s Learning to Performance system is a complete, customized, end-to-end approach. His unique, multi-step process includes: pre-training strategy, planning, design and development, training, and post-training transfer, application and support. The Learning to Performance methodology works with any content - time management, sales, project management - for organizations in any industry. In this dynamic presentation, he will demonstrate how this unique recipe is the key for successful training, transfer, and impact. In this session, attendees will learn how to: Implement the Learning to Performance model. Develop an Impact Map. Integrate instructional design & course development into the learning to performance model. Redesign work process & tools that participants will use when applying their new/enhanced skills. Prepare the management team to ensure successful training transfer. Run classes utilizing the plan of action approach. Implement post-program organizational support by the management team. Launch effective training transfer techniques. Instill post-program application support by trainers.  

Last week I was honored to to have another article published in Chief Learning Officer.  In it I discuss and give pragmatic tips on how to get leaders involved in training efforts.  You can read the article here. I believe I have only touched the surface on how we get leaders involved.  How have you tackled this problem?

Impact Evaluation Evaluation is identifying the direct impact on - and value to - the business that can be traced to training and is part of my Predictive Evaluation model. It assesses in quantifiable terms the value of the training by assessing which Adoptive Behaviors have made a measurable difference. Impact evaluation goes beyond assessing the degree to which participants are using what was learned; it provides a reliable and valid measure of the results of the training to the organization. When participants report a positive impact from training, this approach allows you to articulate how and why training is impacting the business, leveraging this information to enhance the organizational impact of future deliveries. Conversely, when little or no impact is found, this evaluation method uncovers why and determines what can be done to remedy that undesirable outcome. Impact evaluation seeks value to the business. This is done by collecting data on actual business results that participants attribute to successful adoption of their Intention Goal(s). In their Adoption Evaluation survey, participants told us what they did, and we classified them as Successful Adoption or Unsuccessful Adoption. From the successful participants, collect additional impact data via three methods: Completion of an Impact Survey, Interviewing participants, and Examination of company records to confirm findings.  The Impact Hunt It is impractical to follow every participant and determine impact.  So I use sampling with a subset of participants to estimate the impact of the whole population (all participants.) I refer to this as the "Impact Hunt." The method is: Start with all participants and survey them via the Adoption evaluation survey technique. Narrow the potential pool of impact analysis participants to only those who were judged as successfully adopted. Send everyone who is successfully adopted a detailed impact you valuation survey. Using the Impact survey data, identify people who have the highest likelihood or self-reporting impact. This becomes your Impact pool. From this pool randomly sample participants to conducting in-depth impact interview with and evaluate their results. Graphically the Impact Evaluation Process is: As always, please send me your thoughts on this method.  Next blog: Develop the Impact Survey.  

Adoption Evaluation, as an output, sorts participants into either Successful Adoption or Unsuccessful Adoption. Participants labeled Successful needed to meet two criteria: (1) self-reporting successful implementation of their Intention Goal and (2) what they did matches an Adoptive Behavior on the Impact Matrix. These are the participants most likely to have results, and you want to more information from them on that impact. You do this via an Impact Survey. The Impact Survey is like the scorecard of results. From the participants, you want it to capture the following: Details on the performance: what they did, what tools and techniques they used Results realized from that performance: what impact has occurred (cost savings, higher production, less defects, increased sales, etc.) Where claimed results can be validated Percentage by which training provided the impact Whether the impact is sustainable and repeatable Percentage by which other factors (external or internal to the company) contributed to the impact The survey needs to collect enough data from participants so that you can analyze the results and compare them to the predicted impact (found in the training’s Impact Matrix). A good starting point is to review the Impact Matrix and use it as your guide for the survey. SAMPLE PREDICTIVE EVALUATION IMPACT SURVEY As always, please send me your thoughts on this method.  Next blog: Collect Detailed Impact Data from Successfully Adopted Participants.  

Prior to sending the survey, it is a good idea to build the data analysis model that used to summarize the data. It usually takes the form of a spreadsheet or database. In a spreadsheet, the data structure in each row is one participant response with the columns being the data. When to collect data is a factor of the number of course deliveries and volume of participants trained. Other guidelines on the timing or frequency of data collection include the following: Intention Evaluation: collect at the conclusion of each course. For classroom courses, collect a goal sheet from each participant. For e-learning courses, embed the goal sheet into the end-of-course programming and submit it to the data analysis model. Adoption Evaluation: collect data when a sufficient number of participants have graduated and have had the opportunity to transfer their skills to the workplace. If you have a sufficient population size, you do not have to collect adoption data from all participants. Collect enough data to ensure a reasonable return rate and sample to obtain a high degree of confidence. Impact Evaluation: collect data immediately after the Adoption data have been analyzed and reported. Survey Administration Draft an email from a company executive soliciting survey completion and watch the response rate (the ratio of number of people who answered the survey divided by the number of people in the sample, usually expressed in the form of a percentage). A reminder email is sent requesting completion of the survey. You want to get the highest return rate as possible, because obtaining a high response rate can bolster statistical power, reduce sampling error, and enhance the generalizability of the results to the population (participants). Data Scrubbing When collection has ceased, you need to scrub the Impact data. Data scrubbing, sometimes, called data cleansing, is the process of detecting and removing or correcting any information in a database (or spreadsheet) that has some sort of error. This error can be because the data are wrong, incomplete, formatted incorrectly, or are a duplicate copy of another entry (the participant responded multiple times). Simply review the data and make the necessary corrections. As always, please send me your thoughts on this method.  Next blog: Analyze Impact Survey Data. Previous Blogs in the Impact Evaluation Series The Impact Hunt Develop Impact Survey  

Many times evaluators struggle with the balance between using quantitative and qualitative data in training evaluations. I like to use both. However, the element that determines what type of data I use always comes back to the purpose of the evaluation and the questions that evaluation will answer. These two things determine what types of data are needed and how to collect it. It is rare that I do not use both data types. I view quantitative data as the "business end" of the evaluation. Senior executives are used to see data in charts and graphs and like Impact Data in those forms. I use an Impact Dashboard exclusively for these types of data. I always use qualitative data with Impact Evaluation - I view these as the "heart and soul" of the Impact and have found Seniors Executives gravitate more to these than the charts and graphs. What criteria do you use to determine what type of data are used? Quantitative Data Analysis - Relative Comparison Qualitative Data Analysis Read through the data Raw data are ordered and organized so that useful information can be extracted from them. What data do and do not contain. Make notes of the patterns and themes that emerge from the data. Answer these questions: What are the data telling you?  What is missing?  What are the central themes contained in the data?  Are you uncovering something unexpected?  What is it?  Is it pertinent to the evaluation? Quantitative Data Analysis - Measures of Central Tendency and Variability Quantitative Data Analysis - Frequency Distribution Analyzing Qualitative Data Qualitative data analysis involves the identification, examination, and interpretation of patterns and themes in Impact data and determines how these patterns and themes help answer the question of how much impact has been realized. It is important to note that qualitative data analysis is an ongoing, fluid, and cyclical process that happens throughout the data collection stage of your evaluation project and carries over to the data entry and analysis stages. Although the steps listed below are somewhat sequential they do not always (and sometimes should not) happen in isolation of each other. Drawing Conclusions to the Entire Population For each Adoptive Behavior Look at your data and answer this question: What percentage of the sample has successfully adopted? For example, if 50 percent of the sample has been coded as Successful Adoption, you can draw the conclusion that 50 percent of all participants (the population) have successfully adopted. So if 600 participants is the population, 300 would be assumed as Successful Adoption. Next answer: What statistic best describes the results of the sample? Mean, median, or mode? For example, if you judge that the $5,000 for an Adoptive Behavior is representative, choose that value. You then can conclude that the mean value applies to the population (all participants). Calculate the total impact for the Adoptive Behavior by multiplying the statistic chosen (in this example, the $5,000 mean) times the Successful Adoption participants from the population (in this example, 300). Therefore, the total impact for this Adoptive Behavior is $5,000 × 300 = $1,500,000. Note: For a sample to be used as a guide to an entire population, it is important that it be truly a representative of that overall population. Representative sampling assured, inferences and conclusions can be safely extended from the sample to the population as a whole. A major problem lies in determining the extent to which the sample chosen is actually representative. Statistics offers methods to estimate and correct for any random trending within the sample and data collection procedures. There are also methods for designing experiments that can lessen these issues at the outset of a study, strengthening its capability to discern truths about the population. Statisticians describe stronger methods as more "robust." If you have concerns about this, seek assistance from a statistician. Answering the Impact Questions Answer two questions and then (validate your initial findings later, with participant interviews and investigating company records). What impact has the company received from training? Sum all the value being reported and discount by the external contribution factor. You may also view it as an average per participant if that is warranted. How does that impact compare to predicted impact? Look at what you have found and compare it to the course’s Impact Matrix.  Are the results ahead, behind, or on track with the predicted impact? Are you seeing unforeseen impact? Are participants doing things related to training and producing impact that was not predicted? Summary Interpretation is the process of giving meaning to the result of the analysis and deciding the significance and implications of what the data show. Answering the evaluation question means interpreting the analyzed data (quantitative and qualitative) to see if results support or do not support the answers to the evaluation questions. Previous Blogs in the Impact Evaluation Series The Impact Hunt Develop Impact Survey Collect Detailed Impact Data from Successfully Adopted Participants

Join me for a free webinar on August 8th 2-3 PM ET.  In this complimentary webinar, you’ll meet training and evaluation expert Dave Basarab, inventor of the unique Learning Burst training method, which gives participants the opportunity to learn in short bursts anywhere at any time. REGISTER Each Learning Burst covers one topic in a "mini-course" that consists of an 8 to 10-minute audio cast and a PDF workbook that includes relevant simulations, quizzes, interactive exercises, and case studies. To date, thousands of employees from Fortune 500 companies have been taking Learning Bursts as a replacement - or supplement - to traditional classroom training. Participate in this webinar to find out why your company should leverage this cutting-edge training approach, as well. In this session, participants will learn: Determine courses that are candidates for the Learning Burst method. Conduct a Learning Burst needs assessment. Elements in the instructional design unique to the Learning Burst method. How to create a Learning Burst audio segment. The elements in a Learning Burst workbook. How to successfully launch a Learning Burst course. Title: Learning Bursts: A Different Way to Deliver Training Date: Thursday, August 8, 2013 Time: 2:00 PM - 3:00 PM EDT

One element of data collection for an Impact Evaluation involves interviews with participants in order to find examples of the best results and to identify those employees who have no or very little impact. You need only interview a subset of the graduates in order to generalize conclusions to the entire population. From Adoption Evaluation, review the list of participants that you classified as Unsuccessful Adoption: this is the list of potential individuals to interview. Randomly select a sample from both groups. You interview enough people in the High-Impact category to find approximately ten best examples of impact. This may involve interviewing fifteen to twenty graduates or more. Note: the more examples found, the stronger is the correlation from this sample to the entire population. However, these numbers are based on the sample size (the number of graduates). With fifty graduates, it is likely that ten to fifteen are in the High-Impact category, which requires a lower number of interviews. The critical factors affecting interviewing are the time and the budget to conduct the interviews. The more interviews that you can conduct the better, but it needs to be within time and budget constraints. For estimation purposes, a single High-Impact interview (conducting the interview and summarizing the result) takes approximately one hour. Impact Interview Protocol Follow the protocol below when conducting Impact interviews: Inviting High Impact Participants to the Interview Invite participants to the Impact interview via email.  A sample email is: Inviting Low/No Impact Participants to the Interview High Impact Interview Template Low/No Impact Interview Template Previous Blogs in the Impact Evaluation Series The Impact Hunt Develop Impact Survey Collect Detailed Impact Data from Successfully Adopted Participants Analyze Impact Survey Data