July 26th marks the 25th anniversary of the Americans with Disabilities Act of 1990 (ADA) - a comprehensive civil rights law for people with disabilities. The ADA is often called the Emancipation Proclamation for people with disabilities because it ensures that people with disabilities, like all Americans, have "equality of opportunity, full participation, independent living, and economic self sufficiency." The Act prohibits discrimination against people with disabilities in employment (Title I), in public services (Title II), in public accommodations...

I’m Urmy Shukla, Capacity Building Manager at the CLEAR South Asia Regional Center, hosted by J-PAL South Asia at the Institute for Financial Management and Research. Since our 2011 start with CLEAR we’ve developed a wide-range of activities aimed at improving monitoring and evaluation (M&E) capacity throughout the region, including 90 trainings for partners such as the Indian civil services, state governments, NGOs, donor agencies, and academic institutions. Each training requires a significant amount of planning and preparation, including a needs assessment to assess skills and the partners’ role in evaluation, the development of customized content, and delivering the course itself. As such, we want to ensure that are trainings are meeting their objectives.  How do we know if our trainings are ‘working’? As evaluators, we know that there are several steps to plan for, and later assess, effectiveness of our activities. Most importantly, we need to: define a theory of change and/or results framework for program activities, focusing on desired outcomes measure/assess the desired outcomes For evaluation capacity development, these aren’t always easy to design and implement. But we’re taking several steps to assess the effectiveness of our trainings, including developing an organization-specific results framework and tracer surveys to track past training participants. We’re testing our approach as we’re going, and below are sharing some practical and strategic tips. Hot Tips: For training tracer studies: Clearly define training objectives from the outset. These objectives should go beyond skills gained, but should also include what you hope the participants will do after the training, within what is reasonably feasible during that timeline. Develop a way to systematically organize your multiple objectives. This will make it easier for you to design future tracer surveys and needs assessments. We categorize our objectives by (a) partner type (those who either do evaluations, use evaluations for decision-making, fund evaluations, and/or commission evaluations) and (b) knowledge, attitude, or behavior (KAB). From this, we have developed a database of tracer survey questions, which can be easily filtered for each type of training. Get partner buy-in early. Getting people to participate in a tracer study a year or two after the training can be hard, so give advance notice at the training that a tracer study will occur. Then have some contact with trainees - through newsletters, announcements, listservs - after the training to keep contact info current and so they remain familiar with you. Rad Resources: While not M&E focused, this Helvetas Measuring Education’s Path to Prosperity: A practical Toolkit for VET Tracer Studies guide is a useful resource for tracer study planning. Check out our own Interactive Course Manual: A Manual on Organizing and Running a Successful Training Course on Evaluation Methods. The American Evaluation Association is celebrating Centers for Learning on Evaluation and Results (CLEAR) week. The contributions all this week to aea365 come from members of CLEAR. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I am Lycia Lima, the executive coordinator of the newest CLEAR center- for Brazil and Lusophone (Portuguese-speaking) Africa. We’re formally joining CLEAR later this year and are planning our inauguration in October 2015. I was also one of the organizers involved in the formation of the Brazilian M&E Network - Rede Brasileira de Monitoramento e Avaliaçã - which has become a very active association. We’re based in Brazil, at the Sao Paulo School of Economics at Fundação  Getulio Vargas and work jointly with the school´s Center for Applied Microeconomics. Through CLEAR we’re looking forward to expanding into new areas and building bridges with the M&E communities in Brazil and elsewhere. In particular, we’ll be working to advance evaluation capacity development services and products in Portuguese for use in Lusophone (Portuguese-speaking) countries, all to foster evidence-based policy making in these countries. Historically, our team in Brazil has had a lot of experience in carrying out impact evaluations in all sectors. Though we specialize in impact evaluation, we have experience in and appreciate the broader range of M&E approaches, and think that an integrated approach will make our work better. In this post, I have put together a few tips about impact evaluation that you would not learn in conventional econometrics books. This is advice I’d give to impact evaluators. Lessons Learned: Know well the theory of change of your intervention! If you don´t know the theory of change well, you might not fully understand the causality channels and might leave important impact indicators out of the analysis. Get your hands dirty! Go to the field, talk to project managers, talk to beneficiaries and make sure you fully understand the intervention you are trying to evaluate. Also, be careful with the quality of your data. Make sure you spend some resources on hiring and training qualified staff to supervise data collection. Good quality data is crucial for your study. Lessons Learned: Even if you are an empiricist and believe mostly in quantitative methods, do not underestimate the value of mixed methods. In particular, qualitative approaches will help you understand "why and how" things happened. Importantly, get to know M&E "foundational" literature from Patton, Scriven, Bamberger, and others. Rad Resources: While in general M&E materials available in Portuguese are limited in numbers, there is a very useful impact evaluation book that I have co-authored with other Brazilian experts. The book may be obtained free at http://www.fundacaoitausocial.org.br/_arquivosestaticos/FIS/pdf/livro_aval_econ.pdf We look forward to contributing to the M&E literature base in Portuguese, so please check back with us on this. The American Evaluation Association is celebrating Centers for Learning on Evaluation and Results (CLEAR) week. The contributions all this week to aea365 come from members of CLEAR. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.    

I’m Nidhi Khattri from the CLEAR Global Hub at the World Bank’s Independent Evaluation Group.  As a member of the team that got CLEAR up and running, I’ve been interested in how countries make evidence-based decisions, and the role that evaluation plays in that process. Coming from a research-based background where I was more concerned with producing evaluations, I began reading more about how evaluations can be used systematically.  This book on how governments use evaluations to inform budget decisions was especially informative. My understanding grew about the ecology around the production and use of evaluation that is grounded in public sector (or indeed organizational) management.  For evaluation evidence to be used, it’s not enough for the evaluation to be technically sound.  It must be timed correctly and connected closely to the different points of decision in the policy cycle - policy design/budget allocation, program design, implementation, review, and back to budget allocation (both within and across sectors and programs) - and to the fundamental questions that policymakers (and program implementers) must contend with at those specific points in the cycle.   Furthermore, the set of evaluations an organization or a government conducts or commissions must also be based on principles of effective and efficient use of resources, helping guide the choice of evaluations. Many countries (and large organizations) have developed different institutional mechanisms and arrangements to deal with these issues.  They attempt to address the different points in the policy cycle, but they close the loop only partially.  Some focus predominantly on budget decisions.  Others are far more robust in considering and solving implementation issues.  Still others focus much more on accountability at the end.  In part it’s because of issues of coordination and capacity across the range of ministries and departments.  It’s also is due to differences in management philosophy and use of monitoring rather than evaluation.  Similarly, there are quite a few differences in governments (and organizations) making rational decisions regarding the set of evaluations, ranging from somewhat formulaic approaches to letting "…a thousand flowers bloom." This subject - use of evaluation as a tool for public sector management - intrigues me, and I wonder how it’ll evolve in the future with greater access to technology and multiple sources of information, collected and analyzed on an ongoing basis by non-evaluators.  Will it be tied less to decisions at specific points in time and far more to real time decision-making? Will evaluations become less "evaluative" and more "facilitative" along the entire cycle?  In which case the question regarding the choice of a set of evaluations may become moot. Rad Resources: World Bank Independent Evaluation Group’s case study series on M&E systems The American Evaluation Association is celebrating Centers for Learning on Evaluation and Results (CLEAR) week. The contributions all this week to aea365 come from members of CLEAR. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.  

On July 22, @shrmnextchat chatted with the EEOC Commissioner Chai Feldblum (@ChaiFeldblum) and Jonathan Segal (@Jonathan_HR_Law) about the 25th Anniversary of the Americans with Disabilities Act. In case you missed this important chat you can read all the tweets, many which include great links to ADA resources, here:   [View the story "#Nextchat RECAP: The 25th Anniversary of The Americans with Disabilities Act " on Storify]  ...

Hi everyone!  Liz Zadnik here, aea365 Outreach Coordinator and occasional Saturday poster.  Today I’d like to chat about a few people who have shaped the way I approach and appreciate evaluation. Lesson Learned: Feminism is for everybody (…thanks bell hooks).  "The soul of our politics is the commitment to ending domination." I carry this message with me as I engage in my primary role as a trainer and preventionist in the anti-sexual violence movement, as well as when think about evaluation.  How am I dismantling forms of oppression while both engaging in evaluation and talk about evaluation to others? Hot Tip: Don’t be afraid to think with things.  I learned so much working with Sharon Wasco on some community readiness assessment and pilot projects.  Sharon introduced me to so many new ideas as a trainer and evaluation enthusiast.  One of the tools she gave me - and I still use today - is bringing objects and crafts to evaluation training to help adult learners activate new learning styles and energies.  Sharon is also responsible for my deep appreciation for community readiness and the power of qualitative methodology! Rad Resource: Maria Popova of Brain Pickings.  How has this blog helped me with evaluation, you ask?  Well, it’s Maria’s approach to the blog and how creativity shapes our thinking, I think of it as LEGOs — if the bricks we have are of only one shape, size, and color, we can build things, but there’s a limit to how imaginative and interesting they will be. The richer and more diverse that pool of resources, that mental library of building blocks, the more visionary and compelling our combinatorial ideas can be. Lesson Learned: Evaluation is for everybody (…thanks Stephanie Townsend).  Stephanie was the first professional evaluator I worked with; she brought an extensive background in the anti-sexual violence movement and a passion for evaluation I had never seen before.  I owe a lot of "click!" moments to her.  I LOVE spreadsheets and data analysis in large part to her.  She is a brilliant evaluator and patient teacher. This last person is really…well…a group of people.  In no way monolithic, but still someone I carry with me: survivors of trauma and violence.  I started my career about seven or eight years ago working in a community-based domestic violence organization.  I worked with youth in schools and our shelter.  Since then I have talked to countless women, children, and young men who have experienced (and resiliently moved forward from) some of the worst this world has to offer.  My approach to evaluation always has them in mind and I consistently challenge myself to find ways to honor them in this work. Do you have questions, concerns, kudos, or content to extend this aea365 contribution?  Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip?  Please send a note of interest to aea365@eval.org.  aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

  Today's talent pool is better educated and more tech-savvy than ever. They use various technologies as consumers and in their personal lives, and they expect employers to have the same level of comfort with technology as they do. If you talk to someone who was managing an older baby-boomer workforce during the late 1980s or early 1990s, you will likely hear tales of employees who just could not seem to adapt to new technology. You might hear about workers who had difficulty grasping the difference between formatting a floppy and formatting a hard drive, whose typewriters had to be...

My name is Michael Quinn Patton and I am an independent evaluation consultant based in Minnesota but working worldwide. I have had the honor and privilege of participating in and presenting at every Minnesota Evaluation Studies Institute since it began 20 years ago. A lot has changed in evaluation over the last two decades but one thing remains constant: Evaluation is a political activity. The Social Justice theme of this year’s conference highlighted the political nature of evaluation, but politics plays some part in all aspects of evaluation. Lesson Learned: Evaluation is NOT political under the following conditions, all of which must be met: No one cares about the program. No one knows about the program. No money is at stake. No power or authority is at stake. And, no one in the program, making decisions about the program, or otherwise involved in, knowledgeable about, or attached to the program, is sexually active. (Patton, M.Q., 2008, Utilization-Focused Evaluation, p. 537) Hop Tip: Be prepared to deal with politics as a professional The Joint Committee Standards call on evaluators to be politically sophisticated. "Contextual Viability: Evaluations should recognize, monitor, and balance the cultural and political interests and needs of individuals and groups." The AEA Guiding Principles call for evaluators to exercise "Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation." Lesson Learned: Beyond Neutrality Enter the political fray from a strong values base. In a classic article distinguished evaluation pioneer Bob Stake articulated what evaluators care about: We often care about the thing being evaluated. We, as evaluation professionals, care about evaluation. We advocate rationality. We care to be heard. We are troubled if our studies are not used. We are distressed by underprivilege. We see gaps among privileged patrons and managers and staff and underprivileged participants and communities. We are advocates of a democratic society. Rad Resource: "How Far Dare an Evaluator Go in Saving the World?" Bob Stake. American Journal of Evaluation, Vol. 25, No. 1, 2004, pp. 103-107. Lesson Learned: Everybody’s got to serve somebody.  Know whose interests you serve in an evaluation. Not sure about this? Minnesota native son Bob Dylan’s evaluation anthem makes it clear. Check it out. Rad Resource: Bob Dylan singing "Gotta serve somebody" (music and lyrics) Rad Resources: Where Politics and Evaluation Research Meet, Carol H. Weiss, American Journal of Evaluation, February 1993; vol. 14, 1: pp. 93-106. Politics and Evaluation: More Than Methodology, Lois-ellin Datta, American Journal of Evaluation, June 2011; vol. 32, 2: pp. 273-294. Politics and Evaluation, Michael Quinn Patton, American Journal of Evaluation, February 1988; vol. 9, 1: pp. 89-94. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

It is no secret that today’s Human Resources, or Human Capital, industry made its way onto the scene as The Personnel Department. In a time long before sophisticated networks housed human resources information systems, when employee information was maintained on 3×5 index cards in a file cabinet down the hall, we were signing and cutting physical checks for payroll, walking through rows of offices and cubes distributing typewritten memorandums, and a variety of "other duties as assigned" (aka, the admin work no one else wanted to do). ...

Hi, I’m Nora F. Murphy, a developmental evaluator deeply committed to social justice. I recently attended the Minnesota Evaluation Studies Institute (MESI) Spring Training and Donna Mertens’ workshop on Weaving Social Justice and Evaluation Together. Stimulated by the concepts and conversations I have been reflecting on how social justice appears in my practice and had the following insights: Lessons Learned: #1: I actively choose evaluations of projects related to systems change to increase social justice and equity and assumed this was enough. Mertens challenged us to go a step further by placing human rights and social justice at the center. While these elements are always present in my evaluations they are not always at the center. #2: Where people are working towards social justice and equity there is trauma—individual and community, past and present. Evaluators can ignore this and, I suspect, often do. I realized that my most meaningful evaluations did not ignore this but rather recognized and honored this aspect of people’s experiences. #3: AEA’s Guiding Principles For Evaluators (2004) states that evaluators bear responsibility for general and public welfare. When designing an evaluation I can choose to ignore the trauma or design an evaluation that creates the space to recognize the trauma and promote healing as a way to benefit both individuals and society as a whole. I will ask myself these questions and commit to the following as I explore the intersection of evaluation, social justice, trauma, and healing: What gets placed at the center? Mertens suggests we place human rights and social justice at the center. I will be more intentional about doing so. How do I attend to what’s in the center? I will consider methods that promote healing through deep listening, bearing witness, and creating opportunities for people to connect to their inner selves and to each other. For what purpose and to what ends do we evaluate? Bob Williams suggested recently in an EvalTalk post (4.4.15) titled "Evaluation’s Warrant" that one possible purpose is to serve humanity. I will deepen my thinking about this idea. Who is evaluating? Educator Parker Palmer (2009) asks himself: "How does the quality of my selfhood form— or deform— the way I relate to my students, my subject, my colleagues, my world?" In a similar vein I will ask this question of myself as an evaluator and do the inner work needed to bring my best self to my work. Rad Resources: Palmer’s The Courage to Teach: Exploring the Inner Landscape of a Teacher’s Life. AEA365 MESI Week: Donna M. Mertens on Evaluation’s Contribution to Solving Wicked Problems AEA356 CP TIG Week: Karen Countryman-Roswurm and Bailey Patton on Qualitative Research Methods as an Empowering Practice with Marginalized Youth Chilisa’s Indigenous Research Methodologies Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.