The boundaries of an evaluation generally identify what is included within the evaluation.  To identify and state the boundaries for an evaluation, it is required to clearly define items that are inside and outside of the work. All this is about defining the limits and exclusions for the evaluation. Stakeholders should review a drafted boundaries to decide whether a particular aspect should be included in the evaluation or not. Answer this question: What are the specific items that are not within scope of the evaluation?   Example of evaluation boundaries: We will not include non-XYZ course participants. We will not include participants who did not complete the course. We are not doing a Level 4 Results/Impact evaluation. We will not redesign the program or evaluate instructors. Your Quality Check Defines evaluation scope Must be aligned with evaluation objectives Are they complete?

Evaluation Stakeholder management is critical to the success of every evaluation. By engaging the right people in the right way in your evaluation during the planning, it can make a big difference to the evaluation success. A stakeholder is a person or organization that: Is actively involved in the evaluation. Has interests that may be positively or negatively affected by the performance or completion of the evaluation. May exert influence over the evaluation, its deliverables, or its team members. Evaluation stakeholder management is an important discipline that successful evaluators use to win support from others. It helps ensure that their evaluation succeed where others fail. Stakeholder Analysis is the technique used to identify the key people who have to be won over. You then use Stakeholder Planning to build the support that helps you succeed. The benefits of using a stakeholder-based approach are that: You can use the opinions of the most powerful stakeholders to shape your evaluation at this early stage. Not only does this make it more likely that they will support you, their input can also improve the quality of your evaluation. Gaining support from powerful stakeholders can help you to win more resources - this makes it more likely that your evaluation will be successful By communicating with stakeholders early and frequently, you can ensure that they fully understand what you are doing and understand the benefits of your evaluation- this means they can support you actively when necessary You can anticipate what people’s reaction to your evaluation may be, and build into your plan the actions that will win people’s support. The steps to engage evaluation stakeholders are: Step 1. Identify your stakeholders. The first step is to brainstorm who your stakeholders are. As part of this, think of all the people who are affected by your evaluation, who have influence or power over it, or have an interest in its successful or unsuccessful effort. Step 2. Prioritize Your stakeholders. Map out your stakeholders and classify them by their power over your work and by their interest in your work. The top people are your key stakeholders. Step 3. Gather key stakeholder requirements. You now need to know more about your key stakeholders. You need to know how they are likely to feel about and react to your evaluation. You also need to know how best to engage them and how best to communicate with them. Key questions that can help you understand your stakeholders are: What financial or emotional interest do they have in the outcome of the evaluation? Is it positive or negative? What information do they want from you? What questions do they want answered from the evaluation? How do they want to receive information from you? What is the best way of communicating your message to them? A very good way of answering these questions is to talk to your stakeholders directly - people are often quite open about their views, and asking people’s opinions is often the first step in building a successful relationship with them. Once you have completed your stakeholder analysis, you can continue with creating your evaluation plan.

Provide evaluation data that is exactly what the business needs, that is actionable by key decision-makers, and that has been interpreted and analyzed, with solid business recommendations.   My latest article was published in Training Magazine on using evaluation to build strong relationships with your business partners. You can read the article here.  I would enjoy hearing your reaction and additions.

By doing all the previous actions in creating an evaluation plan (mission, objectives & deliverables, questions the evaluation will answer, assumptions, boundaries, and stakeholders), you can now develop the strategies you will use to perform the evaluation.  An evaluation strategy describes how the ends (goals) will be achieved by the means (resources).  This details the actions you will perform during the evaluation. Items I usually include in a training evaluation strategy are the methods to, collect data, analyze it, and report the findings.  Here is an example of recent evaluation plan strategy I did for a Level 3 evaluation: Validate the strategies with your evaluation stakeholders and client, make changes as needed, and move to the next step.

What makes PE different? Predictive Evaluation (PE) is the only training evaluation approach that adds the element of forecasting (prediction). Using the PE Model companies can successfully predict training’s results, value, intention, adoption and impact, allowing them to make smarter, more strategic training and evaluation investments. The Predictive Evaluation Model is a practical step-by-step training evaluation process - it’s unique because it: adds the element of prediction to training evaluations. Requires student participation, making associates feel invested and engaged throughout the process. Provides recommendations for continuous improvement. Predicts the quantifiable impact of training to forecast training investments, results, and ROI. While performing all the traditional training evaluation activities. How PE Differs from Other Approaches

The following article was first published on January 23, 2008. Dear Crucial Skills, Someone stole money from me and I have a hunch it was a roommate. How would you approach this confrontation? Our relationship is neither strong nor bad, just fairly new. I’m not sure how to ask her without making her feel unsafe. And I definitely can’t imagine her saying "yes" even if she really did take the money. What should I say? Signed, Baffled Dear Baffled, I sympathize with your situation. Something bad has happened. You can’t generate any plausible explanation other than theft. And yet, it’s hard to see this new roommate as a thief. One of the hardest times to motivate yourself to speak up is when you aren’t whipped-up in righteous indignation. You doubt yourself and you don’t want to cause pain to a potentially innocent person. On the other hand, this is also the best time to speak up because you are in exactly the right frame of mind for real dialogue. You’re humble enough to be wrong and caring enough to worry about the impact of your approach. Of course, what you do depends upon the strength of the story you’re currently telling yourself. So I’ll offer some advice for three scenarios. You choose which fits: 1. No evidence. The only reason you’re even thinking your roommate may have taken your money is by process of elimination. In other words, you don’t think she stole it but you can’t think of any other explanation. In this circumstance you should bring up the missing money. Share the facts—not your story (that you wonder if your roommate stole it). If your roommate had nothing to do with it, this will help involve her in the search or alert her to problems that could continue to plague both of you. Simply say something like, "Last night, I had two $100 bills in my purse. I left it in the kitchen and this morning they were gone. Have you had anything come up missing recently?" If your roommate was involved, this conversation will either put her on notice that you’re aware of something fishy or lay the groundwork for a future, more direct, conversation. But, I don’t recommend this very vague approach if you have more reason to suspect your roommate. 2. A little more evidence but a lot of fear. You have a number of reasons to suspect her (e.g., she had two $100 bills when you went out to eat last night) but have reasons to believe a conversation would do more harm than good (she has a hot temper and carries a Taser). In this situation, you’ve concluded that the potential upside of a conversation is not worth the downside risk of conflict. The big mistake people make in this situation is indecision. They waste time feeling resentful about reality rather than simply accepting their own assessment and making a hard choice to either a) adapt to the insecure environment by securing your valuables; or b) move. Get over it—if you’ve decided you aren’t going to speak up, accept responsibility for that choice and decide how you’ll deal with the future. 3. A little more evidence but nothing to lose. You have a number of reasons to suspect her and nothing to lose by trying the conversation. The worst that can happen is that she denies it, resents you, and you move out. The only difference from the second option is that you’ve opened up the possibility for her to acknowledge her actions and for you to come to some resolve. Here are some ideas for holding the conversation. Don’t open your mouth until you’ve committed to Plan B. Decide what you’ll do if either she denies it and you’re still suspicious or she denies it and the relationship sours. If you’re prepared for this eventuality, you’ll feel a bit less stress in the conversation. Begin with a sincere and emphatic apology. "I have a concern and I feel terrible about even bringing it up. But I know if I don’t, it will nag and bug me and get in the way of our relationship. May I talk with you about it?" Take her carefully down your path to action. Carefully and non-judgmentally share your data. Take all the time you need and don’t skip any element of what feeds your concern. Then, very tentatively, share your conclusion. "The other night I had two $100 bills in my purse when I left it on the counter. I know I did because I opened my billfold to remove $5 for cab fare when I got home. The next morning it was gone. I racked my brains to think of what could have happened to it. Then when you and I went out to eat that night you had two $100 bills." Acknowledge your suspicion but be tentative. At this point she knows what you’re leading to. You must very quickly restore safety in two ways: 1) by letting her know you hate this conclusion—even though you worry about it; and 2) by letting her know if she made a mistake you can still respect her. "I know this sounds horrible for me to even ask. But can you see why I’d be wondering? Since I can’t come up with any other explanation about how it could be missing, I decided I needed to talk to you rather than leave it festering between us. And I want you to know if you did make a mistake, I’ve done so in my life too." Open the dialogue. Now it’s her turn. "Did you—for any reason—take the money from my purse?" Be prepared for her to be hurt and defensive. If she is, do not back down. Continue to ask her to help you reconcile the concerns while assuring her all you want to do is work it out. This is tough, but the costs of not speaking up will be much higher than the risks of taking action now. Be humble and honest and you’ll have done all you can. Finally, if you decide to leave, do so quickly and graciously. When you refuse to let others paint you as a villain, you enable them to examine themselves rather than justify their transgressions using your vengeful response. Best wishes, Joseph

Overcoming adversity is part of the human story. My father was born on the steps of a barn in a poor remote village in Greece. There’s awe in listening to him talk about his childhood and his memories of World War II coming to his small village. I think every generation tells a story that can become the stuff of legend. I grew up in a great ethnic working class neighborhood in Chicago and really did walk a mile every day to school. Rain or shine. Chicago winters can be brutal and those walks seemed like Shackleton’s expedition in the Antarctic when I was younger. You can imagine how those stories have grown when told to my children these days. The hyperbole is entertaining. But part of what is so appealing about my father’s story or the story of countless other older Americans is that it is human nature to want to triumph over adversity. That’s why the lesson I hope businesses and individuals alike can carry away from the current economic crisis is one of resilience. Many of today’s business leaders recognize resilience as the characteristic that makes it most likely for businesses to thrive in a down economy. Resilience is the ability to overcome adversity. The more resilient a business, the more quickly it may recover from a new threat. The economic crisis has had as profound an effect on many people and businesses as a natural disaster. The skills and techniques used to cope with natural disasters can be put to good use dealing with the economic crisis. The American Psychological Association (APA) recently published an article offering advice on how to address distress or anxiety in the face of adversity. The suggestions made by the article are applicable when considering how to handle the challenge posed to businesses by the current economic crisis. The APA recommends five strategies to cope with disaster, which I believe encourage resiliency. The strategies are excerpted below with my comments regarding how the strategies can be applied to the business world. 1. Get the facts. Having the facts about a threatening situation can lessen uncertainty and anxiety. In the business world, it is a leader’s responsibility to get the most accurate, up-to-date information about the threat of possible lay-offs or reorganization and share this information in a timely and carefully planned fashion with his/her employees. Uncertainly is counter-productive. Having all the facts about a given situation also allows organizations to plan ahead to meet the challenges of a down economy. 2. Have a plan. Consult with coworkers to strategize for the changing market conditions. Plan for both the short- and long-term. Take the time to make good decisions about the future success of your business. Don’t just react to potential threats. Consider, for instance, your bench strength and identify the employees you’ll most want to have on board when business starts growing again. Create a plan to keep them engaged. 3. Make connections. Don’t try to weather tough times alone. Share both your fears and your hopes with family, friends, and co-workers. Business leaders should take responsibility for modeling good communication about how the organization can weather difficult times. Engage your employees in strategizing for future success in spite of the down economy. 4. Maintain a hopeful outlook. My favorite stories have the protagonist triumph over adversity. Be the protagonist of your own story. Plan to triumph over adversity. Imagine yourself successful and share your optimism with others. Consider how you have dealt with past hardships and apply that knowledge to the current economic situation. 5. Volunteer. Share resources and best practices. Help your co-workers outside of the workday - especially those who are struggling with fewer resources. Be cognizant of the fact that although everyone may not be equally stressed by the down economy, everyone feels the impact in some way or another. Rather than consider your hardship to be worse, give back to your workplace and your community. For those of you who are rising to the challenge of the economic downturn, I congratulate you. I hope that you have begun collecting stories about how you and your organization overcame adversity - these stories can become the legend of your company’s history. I believe that there is an extra measure of pride in resilience. Rather than just weather the storm, plan to come out ahead; poised and ready for a rebounding market.

The value of assessments is universally recognized throughout the business world. Assessments can be applied at the individual, team and organizational level. Further, assessments are great tools as components of your selection process and development programs. Each application is unique and requires certain safeguards to ensure that data is interpreted and engaged appropriately. Selections Process In a Selection Process we find that there are stars in every company. However, understanding the unique differ¬ences between top, middle and low performers is essential in build¬ing selection criteria and tools. When designed correctly you have a roadmap to hire the right people, and focused development capabilities that will improve individual performance. People that thrive in one company do not always succeed in others. Measuring the differences between high and underperforming talent creates accurate tools that quickly identify the right talent for both the organization and the role. The outcome must be a clear set of compe¬tencies and selection tools that improve hiring decisions and better pre¬dict who will perform in the role, and screen out those who are less likely to be successful. Key steps in the selection process include: Identifying top and bottom performers Conducting one-on-one interviews Profiling Top/Middle/Low performers Identifying discriminating factors Defining selection and development competencies Developing targeted selection tools (assessments, interview questions, etc.) Integrating selection tools into the existing hiring system Measuring and monitoring results to improve the benchmark over time and as the role evolves or changes Development Tools From a Development Tool perspective we believe that each individual has a unique Talent Pattern, which is comprised of multiple dimensions that must be evaluated both individually and based on their integrated assessment pattern. We have found that an individual’s Talent Pattern includes preferences that result in both strengths and potential limitations. Potential limitations come from over-reliance on our strengths, resulting in tendencies, biases and blind spots. Personal effectiveness results from being aware of your Talent Pattern so you can maximize your strengths and manage your potential limitations. Combinations of these Talent Patterns can result in either positive or negative effects, depending on the job requirements and the culture of the organization. These combinations can be described as: Synergizers - Motivator, Thinking, and/or Behavior dimensions with opposing characteristics which, when integrated, result in a total effect which is greater than the sum of the individual effects. Amplifiers - Motivator, Thinking, and/or Behavior dimensions with similar characteristics which, when integrated, result in a magnification of the individual effect. Through years of practice, identifying development opportunities must include evaluation of various dimensions. We recommend examining a person’s talent pattern from at least three perspectives. We first recommend evaluating an individual’s Motivational Style so you can better understand WHY they do what they do. We then recommend evaluating an individual’s Thinking Style, which includes experiences, references, education, and training, and will tell you WHAT they can do. We finally recommend evaluating an individual’s Behavioral Style, which tells you HOW they will do it. Assessment Survey Tools Alignment surveys measure gaps in the perception of customers, employees, managers, and executives of various objective success criteria. Time and again we have found that alignment is key to performance. Often we find that the perception of management is not consistent with the perceptions of team members. More importantly, misalignment between customers and an organization can help identify and isolate underperformance and customer satisfaction challenges. Alignment surveys are a great alternative to 360s. We’ve all probably have participated is some form or other in 360s. Alignment surveys allow organizations to probe even deeper to understand strengths and limitations of their leaders, teams and the organization, and are able to do it in a less threatening and obtrusive manner. Leadership Effectiveness Surveys (LES) provide a quantitative analysis of the culture of an organization and the effectiveness of its leaders. They also assess the ability of leaders to drive employee performance and engagement. LES are often a key component of leadership development programs as it allows leaders and the organization to measure improvement and clearly identify ROI and provide guidelines for areas of continued development. Assessments and surveys can be integral tools in selection, development and leadership applications within your organization. When applied appropriately, they bring great value, increased performance and a winning culture to all aspects of your employees, leaders, teams and organization.

Leaders today are caught in times of unprecedented organizational change and complexity. Whether their focus is on bottom-line results, cost reductions, change efforts, globalization or IT - the pressures are immense. Leaders are dealing with loss of employee and public trust, the loss of valuable talent, all while expected to do more with less. In the midst of all this, they are also expected to be proactive in developing their abilities and to recognize the importance of developing the people around them. Coaching is not a last ditch effort to "fix" someone or a focus only on life issues. It’s about leaders having an impact and producing results. It’s about building leadership capacity and improving how a leader functions within your environment. Coaching should incorporate stakeholder feedback, align leaders with development objectives and hold them accountable for their development, while measuring progress throughout the engagement. In the end, coaching should help develop effective leaders that will create a winning culture within your organization. A recent Harvard Business Review research report found that most coaching engagements fall into two categories: developing the capabilities of high-potential leaders or facilitating a transition. This demonstrates that coaching today is less about fixing problem behaviors and more about enhancing the performance of valued executives. Other key areas of the coaching engagement include: * Facilitating a transition * Developing capabilities of a high-potential manager * Acting as sounding board on organizational dynamics * Enhancing the interactions of a team * Addressing a "derailing" behavior * Act as sounding board on strategic matters The HBR report stated that executives who are willing to learn and evolve and who can actively engage in the coaching process have the best results. Leaders with significant character flaws or deep-seated behavioral problems are poor coaching candidates. People suffering from deep resentment, severe narcissism, ironclad beliefs, or a victim mentality are usually unwilling to look inwards and are therefore unlikely to benefit from coaching. Success of the coaching engagement depended on the quality of the relationship between coach and leader. Further, the support of the company was also deemed very critical. Firms must be committed to the leader and his or her progress, and senior management needs to be invested in the engagement. There are two basic rules for hiring a coach. First, make sure that the executive is ready and willing to be coached. Second, allow the executive to choose whom he or she wants to work with, regardless of who in the organization initiated the engagement. Organizations should also take into account whether the coach has a clear methodology. Different coaches value different methodologies. Some coaches begin with 360-degree feedback, for example, while others rely more on psychological feedback and in-depth interviews. From an organization’s perspective, methodology is a good way to winnow the pile. If a prospective coach can’t tell you exactly what methodology he uses - what he does and what outcomes you can expect - show him the door. Top business coaches are as clear about what they don’t do as about what they can deliver. For example, a good coach will be able to tell you up front whether or not she is willing to serve as a sounding board on strategic matters. All coaches recognize that they should be making you more competent and self-reliant. If the coaching relationship isn’t doing that, it’s very likely that you’re becoming overly dependent. Dependence isn’t always bad, of course - friends relying on one another, for example, is a good thing. But we all know people who can’t make a decision without first talking to their psychotherapists, and some executives defer to their coaches in the same way. They have conversations with the coach that they ought to be having with other executives in the C-suite or with their teams. Coaches have an economic incentive to ignore the problem of dependency, creating a potential conflict of interest. It’s natural for them to want to expand their business, but the best coaches, like the best therapists, put their clients’ interests first. Harry Levinson, the father of coaching, worked with the top executives of his day. He said that if a coach wasn’t aware of the dependency dynamic, then he had no right to be a coach. What this means for you is that before you hire a coach, you should ask her how she handles dependency in relationships. Coaches can also be very lax in evaluating the impact of their work and communicating results to executives and stakeholders. While it can be difficult to draw explicit links between coaching intervention and an executive’s performance, it is certainly not difficult to obtain basic information about improvements in that executive’s managerial behaviors. Coaching is a time-intensive and expensive engagement, and organizations that hire coaches should insist on getting regular and formal progress reviews, even if they are only qualitative or anecdotal. There’s no question that future leaders will need constant coaching. As the business environment becomes more complex, they will increasingly turn to coaches for help in understanding how to act. These coaches will do more than influence behaviors; they will be an essential part of the leader’s learning process, providing knowledge, opinions, and judgment in critical areas.

For many companies, innovation is their lifeblood.  Without innovation, it is difficult to remain competitive in a rapidly changing marketplace.  But sometimes it can seem like we’re always chasing after the next best thing instead of recognizing what we’ve accomplished.  I know a lot of managers would tell me that there isn’t time to stop and smell the roses.  And to some extent I agree.  I believe that unless and until continuous improvement is instilled as a way of life in an organization there is a risk of stagnation.  Just because a product is good doesn’t mean it can’t be great.  And just because a product is good doesn’t mean it will still be in demand two, five, or ten years from now.  Savvy leaders know to encourage innovation in the workplace.  The cautionary tale is not to innovate for the sake of innovation.  It’s always important to maintain best practices and processes and not sacrifice product quality and control in the search for something new or better. It can be exhausting to always be expected to be on the verge of the next best thing.  Creativity doesn’t always flow easily, which is why it is important for leaders to put in place certain processes to aid innovation.  Good leaders encourage teams and divisions to share best practices.  A leader can encourage healthy competition within an organization and still stress the importance of sharing new processes, ideas, and/or technologies that will help others in the organization to improve their performance.  Good leaders also see the value in sharing best practices with others in the industry.  They are secure enough in their products and services to know that sharing a best practice with the competition can be mutually advantageous.  Mike recognized this during his career as captain of Benfold.  At first, he focused on making Benfold the best damn ship in the Navy.  He later realized that he wanted the U.S. Navy to be the best damn Navy in the world, which meant sharing best practices with other captains. Good leaders can formalize innovation by challenging the status quo and raising expectations on a semi-regular basis.  They can also recognize the benefit of taking calculated risks.  There is no guarantee that a new process or technique will succeed the first time.  In fact, those of you involved in software development would likely assure us that it’s more likely to fail than succeed.  But every failure teaches us something new.  Every failure is, in itself, an innovation.  I believe that a workplace culture should encourage innovation and allow for failure.  It should also celebrate the efforts of its employees towards improving products and services.  And, finally, innovation should be viewed as a means to an end and not the end itself.