My name is Judah Viola and I am a Community Psychologist at National Louis University in Chicago. I also manage an independent consulting practice specializing in program evaluation and collaborative community research. Whether boom or bust economy, it is never a bad time to build your own evaluation skills and capacity to provide value for clients. Fortunately, there is a plethora of information available about the art and science of evaluation. However, there isn’t much published or taught in universities about how to build your evaluation consulting practice. Knowing what analyses are appropriate for a particular project and how to conduct them are necessary, but you also need to be able to translate statistics or methodology into language that all stakeholders can understand and utilize for decision making. In addition, you need to know a whole host of other things such as how long the project will take and how much the evaluation is worth to the client. Thus, for the majority of evaluation consultants, academic training alone is not sufficient preparation, and real word evaluation experience working under others is the preferred option. Hot Tip: Start with your existing network. Who introduced you to the field of evaluation? Many faculty do evaluation projects and look for students, or early career evaluators to support their work. In addition, many independent consultants subcontract parts of their larger projects out to colleagues and are willing to supervise and train folks with little experience. If you don’t currently know many evaluation consultants, there are simple ways to build your network. Hot Tip: Getting involved in local professional development organizations, promotes learning and networking, and opportunities for referrals or collaborations. My involvement in the Chicagoland Evaluation Association (CEA), a local AEA affiliate, has enabled me to connect with evaluators with whom I have worked on evaluation projects, presented at conferences, and published book chapters. Hot Tip: Contact and listen to experts on the phone, on-line, or at conferences. Initially, I thought that experienced independent evaluation consultants may be too busy or hesitant to talk to a newbie. However, I found that they (especially AEA members) are quite approachable and generous with their time and are quick to share lessons learned so you don’t have to make the same mistakes they did when getting started. I’ve gotten a host of great advice about everything from getting an accountant to how to come up with a pricing strategy. Rad Resources: Consulting and Evaluation with Nonprofit and Community Based Organizations by Judah Viola & Susan McMahon (2010) Independent Evaluation Consulting: New Directions for Evaluation, No. 111 (2006) Evalbusiness: The Independent Consulting TIG listserv This post is a modified version of a previously published aea365 post in an occasional series, "Best of aea365." Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Judah Viola on Building Capacity to Succeed as an Independent Consultant Heidi Gegax on Creating a Consultants Collaborative IC Week: Jan Upton on The Independent Consultant Life Cycle

My name is Käri Greene and I’m a Senior Research Analyst at Program Design & Evaluation Services, an intergovernmental agency for the Oregon Public Health Division and Multnomah County Health Department, as well as a co-Chair for the LGBT Issues TIG. Hold on…what was that jumble of letters at the end of the sentence? Well, our TIG explores areas of sexuality, gender and identity as they relate to evaluation theory, practice, and use, specifically focusing on issues related to lesbian, gay, bisexual, and transgender issues. Many evaluations might not deal explicitly with LGBT issues; however, gender and sexuality are concepts present in much of our evaluation practice. Gender or ‘sex’ is a standard demographic variable collected in nearly all evaluation studies, and sexual orientation is being included more frequently in evaluations. But the concepts of sexual orientation, sexual behavior, and gender identity can be dynamic and complex. In public health evaluations, someone served by a program might identify as a lesbian woman, but she may have been born and raised as a boy and not identify as transgender. A man served at the local public health clinic might be having sex with other men, but not identify as gay or bisexual. Being clear about what we need to know about the clients served in our programs is essential to answering our evaluation questions. 2015 Update The key thing to keep in mind when dealing with issues of sexuality and gender is to question assumptions and ask the right questions for your evaluation and those served by the program. Sexual orientation does not automatically define a person’s sexual behavior, and gender identity does not always fit neatly into a two-by-two table. Feeling even more confused about how to deal with gender and sexuality? That’s good - that means you’re questioning assumptions! But it can be frustrating. The field is evolving and even after a century of research on sexuality and gender, few researchers agree on terminology, dimensions and categorical classifications of sexuality. But fear not, we’ll have more to say on this subject throughout the week… Hot Tip: Consider how you currently assess gender. It might be important to ask multiple items to get at gender - one that asks current gender identity ("Do you consider yourself to be male, female, transgender, or something else?") and one that asks birth gender ("What sex were you assigned at birth - male, female, or intersex?"). Hot Tip: Consider expanding your existing response categories for sexual identity. Younger clients might consider themselves "queer" as opposed to the more traditional categories of lesbian, gay, or bisexual. 2015 Updates Rad Resource: The Williams Institute "Best Practices for Asking Questions to Identify Transgender & Other Gender Minority Respondents on Population-Based Surveys" Rad Resource: The Gay & Lesbian Alliance Against Defamation (GLAAD) has resources, including a media reference guide, that can be helpful when communicating and reporting about issues of sexuality and gender. Rad Resource: "Do Ask, Do Tell" article by Cahill et al. on the acceptability of asking patients sexual orientation and gender identity in clinical settings. This post is a modified version of a previously published aea365 post in an occasional series, "Best of aea365." Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: LGBT Week: Käri Greene on Issues of Gender and Sexuality in Evaluation LGBT Week: Joseph Kosciw on Making Schools Safe and Affirming Places to Learn for LGBT Students LGBT Week: Denice Cassaro on Using Open-Ended Demographic Categories to Learn About Race, Ethnicity, Sex, and Gender

Hi there, Liz Zadnik here, new(ish) member of the aea365 curating team and sometimes Saturday poster. Last year Sheila posed the question What is it that YOU would like to read about on this blog? One of the responses resonated with me, as it represented my relationship with evaluation as a professional: I would love to see a post, or series of posts about evaluation from the perspective of practitioners for whom their primary job is not evaluation. Perhaps tips on how to best integrate evaluation into the myriad of other, seemingly more pressing, tasks without pushing it to the back burner. I work in the anti-sexual violence movement at a state coalition, focusing on prevention strategies, training, and making community-based rape crisis centers accessible to people with disabilities. These three areas are my priorities - there are deliverables and activities that don’t always include evaluation and assessment. Many times - given my love of evaluation - I am the sole voice at the table asking about an evaluation plan. Most of the time we can weave evaluation in from the ground floor, other times it happens a little late(r). Hot Tip: Ask this (or a similar) question: "How will we know we’ve been successful?" This is the most effective way I have found to help get people thinking about evaluation. It has started some of the most engaging and enlightening conversations I’ve ever had, both about a project and the work of the movement. Lesson Learned: Sometimes, evaluation takes a backseat to program implementation and grant deliverables. This can be disappointing (to say the least), but I do see a change. Funders are more frequently asking for research, "evidence," or assessment findings, providing evaluation enthusiasts (like myself) to engage our colleagues in this work. Lesson Learned: Practice and challenge yourself, even if no one is ever going to see it. One of the ways I "integrate evaluation into the myriad of other, seemingly more pressing, tasks" is evaluating myself and my own performance. I regularly incorporate evaluative questions into training feedback forms, look for ways to assess the effectiveness of my technical assistance provision, and record my professional progress throughout the year. I sit in on as many AEA Coffee Break webinars and other learning opportunities as I can, always practicing the skills discussed and looking for ways to apply them to my work. I would so appreciate hearing from other practitioners (and evaluators!) about their experiences infusing evaluation into their work. I’d also be happy to answer any questions you might have or write about specific projects in the future. Let me know - the aea365 team is here to please! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Sheila B Robinson on A Call for Blog Posts! Jara Dean-Coffey on Visual Facilitation and Graphic Recorders IC Week: Norma Martinez-Rubin on Working Solo vs. Subcontracting

Hi. My name is Lacy Fabian, and I’m a healthcare quality researcher with MITRE in Woodlawn, MD. In addition to my work in healthcare quality, I am a preceptor for advanced pharmacy students. As a board member for the Eastern Evaluation Research Society (EERS), my AEA local affiliate, it’s my pleasure to welcome you to EERS week on AEA365. This week, some of our members will be sharing lessons learned, hot tips and rad resources in the evaluation field. As tends to be the case for many, I came to evaluation via an indirect path and professional societies like EERS and AEA have been mainstays in my continuing professional development, as well as outlets for giving back to the field. This has been particularly true in learning how to talk about evaluation with others from colleagues to clients. "What is evaluation?" continues to be a challenge, as many are only familiar with a small branch of evaluation, if at all. Lesson Learned: In the last few years the dialogue around evaluation has improved. People are increasingly aware of the presence of evaluation in their daily lives from education to healthcare to spending on government programs. That type of shift doesn’t occur only at the programmatic and policy level, but also through personal interactions. In meetings or workgroups there are often opportunities for an evaluator to employ their methods to solve challenges at hand—think process maps or logic models. Lesson Learned: Interacting with the field via a niche or sweet spot that you’ve identified helps connect evaluation to new areas and build understanding of what it can offer. For example, you may present your evaluation findings at subject matter conferences, mentor students, give a guest lecture on the importance of evaluation to a particular discipline, or volunteer your expertise to a community effort that resonates with you. Hot Tip: Take advantage of daily evaluation resources, whether it be through AEA blogs, journal article alerts or LinkedIn groups—to see the field as a whole and when shifts or challenges are occurring. Rad Resource: Registration is open for the 38th Annual EERS Conference, which will be held April 26-28, 2015. The theme is "Let’s Get Real: Evaluation Challenges and Solutions," and we hope to see you there to exchange ideas about your evaluation lessons learned. The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Leigh M. Tolley on Student Involvement in Local Affiliates Susan Kistler on Local Affiliates and Managing Professional Obligations MNEA Week: Danielle Hegseth on A New Kind of Education at the AEA Conference

Hi! We are Jill Feldman and John Kelley, board members of the Eastern Evaluation Research Society (EERS), AEA’s oldest affiliate, writing to tell you about the Eleanor Chelimsky Forum, a hallmark of our annual EERS conference. Eleanor Chelimsky is one of the most insightful, influential, and respected evaluators of our era. At the 2012 EERS Conference, Eleanor proposed the idea of a forum to spark debate using common practitioner experiences that challenge evaluation theory as a way to improve both theory and practice. When Eleanor speaks, EERS and the Robert Wood Johnson Foundation (RWJF) listen! The inaugural Eleanor Chelimsky Forum was held in 2013. The Forums are funded through a generous grant from RWJF. Lesson Learned: Attracting high caliber speakers to the Forum was a pleasure not a problem. Michael Quinn Patton and Tom Schwandt, Deb Rog, and Abe Wandersman readily agreed to kick off EERS conferences in 2013, 2014 and 2015, respectively by serving as featured Chelimsky Forum speakers. Rad Resource: Manuscripts written by Chelimsky Forum speakers are published each year in the American Journal of Evaluation (AJE). The 2013 inaugural articles by Michael Quinn Patton and Tom Schwandt and discussant response by Laura Leviton of RWJF are available on AJE’s website. Stay tuned for Deb Rog’s soon-to-be-published article based on her 2014 address. Rad Resources: Posting videos of the Chelimsky Forum on the EERS YouTube channel helped spread the word as did joining forces with AEA to create a webinar with Deb Rog that reached evaluators across the globe. Rad Resource: Read Eleanor Chelimsky’s 2013 article, "Balancing Evaluation Theory and Practice in the Real World." Hot Tip: Don’t miss the 2015 Chelimsky Forum when Abe Wandersman delivers his plenary address "Achieving Outcomes in a Specific Project: Why Evidence Based Interventions are not Sufficient and Why Evaluation is Key" with remarks by discussant Mary Dixon Woods. The 2015 EERS conference will be held April 26-28 in Galloway, NJ - just outside of Atlantic City. Watch for the YouTube video of the Chelimsky Forum to be posted shortly thereafter. The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Susan Kistler on the AEA Thought Leaders Forum Leigh M. Tolley on Student Involvement in Local Affiliates EERS Week: Ann K. Emery on Overhauling Your Organization’s Data Visualizations? Three Edits Guaranteed to Give You the Biggest Bang for Your Buck

By Christine Schaefer Who are the folks who judge applications for the Malcolm Baldrige National Quality Award? In an ongoing blog series, we have been interviewing members of the 2015 Judges’ Panel of the Malcolm Baldrige National Quality Award. In the interviews, they share their insights and perspectives on the award process, on their experiences, and on the Baldrige framework and approach to organizational improvement. Following is the interview of Fonda Vera, a second-year judge. Vera is executive dean of planning, research, effectiveness, and development at Richland College (PDF), the first and only community college to date to receive a Baldrige Award (in 2005). What experiences led you to the role of Baldrige judge? I began my involvement with the Baldrige Program as an applicant in 1999. I was one of the writers of [Richland College’s] numerous state and national applications. Just writing the application and studying the Criteria from that point of view was a wonderful learning experience, not only in learning about the Criteria but also about my own organization. After we were [named] a Baldrige Award recipient in 2005, I became a national examiner. I was fortunate enough to go on a site visit the very first year and also for the next two years in a row. In total, I have participated in four site visits, and I have led one. It was an incredible honor to be chosen to sit on the panel of judges. You have a great deal of experience in the education sector. How do you see the Baldrige Excellence Framework as valuable to education organizations? The Baldrige Excellence Framework is invaluable to education. The education sector is buffeted about by ongoing state and national legislative changes, calls for accountability demanding more performance with less funding, and students who are less and less prepared—for a multitude of reasons—for success in education and ultimately in life. The Baldrige Excellence Framework is the rudder for the education sector in that stormy sea. It empowers organizations to address their reason for being by maintaining focus and discipline to achieve student success. How do you apply Baldrige principles/concepts to your current work experience/employer? Richland College has used Baldrige principles and the framework since 1999. It quickly became the way we do business. We have been fortunate to have consistency in leadership, which I believe is key. This discipline of following the Criteria is not always easy, but it is hard to argue with the good results it produces. Our college’s vision of being the best place we can be to learn, teach, and build sustainable local and world community fits perfectly with the Baldrige core values and concepts. As a judge, what are your hopes for the judging process? In other words, as a judge what would you like to tell applicants and potential Baldrige Award applicants about the rigor of the process? As a current judge, a previous applicant, and a previous award recipient, I would say to potential applicants that following the Baldrige Criteria and writing an application is one of the best things you can do for your organization. It will cause you to examine what you do, why you do it, and for whom you do it in a way that you’ve never done before. Using the Socratic Method, this deep dive into your organization’s business will reveal opportunities to improve and strengths you may never have known existed. The challenge and the rigor for each organization is to have the discipline to take what is learned, act on it, and invest in the discipline for the long haul. The rewards are many if you follow this path. What encouragement/advice would you give Baldrige examiners who are reviewing award applications now? I encourage all Baldrige examiners who are reviewing award applications to do their absolute best to understand the applicant they are reviewing and to make comments that are relevant and will move the applicant to the next level of maturity. This is not only important to the applicant but also is vital to the judging process. While the examining process is certainly very hard work, take the time to enjoy the intellectual challenge and the camaraderie that accompanies it. It is indeed a unique experience. See other blogs on the 2015 Judges’ Panel: Laura Huston, Dr. Ken Davis, Michael Dockery, Miriam N. Kmetzo, Dr. Sharon L. Muret-Wagstaff, Dr. Mike R. Sather, Ken Schiller, Dr. Sunil K. Sinha, Dr. John C. Timmerman, and Roger M. Triplett. Greg Gibson, a candidate for the 2015 panel, pending appointment, will also be interviewed for this series.

Greetings, I’m Ann K. Emery. I consult, instruct, and write on all things data visualization. One of the most common questions I receive during workshops and webinars is, "Ann, where do I start?" Cool Tricks: As you’re overhauling your visualizations, these three edits are guaranteed to give you the biggest bang for your buck. Remove unnecessary ink. I immediately begin deleting or lightening everything without a purpose: the border, the grid lines, and the tick marks. Visualization guru Edward Tufte calls this strategy the data:ink ratio; we’re intentionally removing any ink that isn’t directly related to the data itself. These edits ensure our viewers will focus attention where we need it: on the actual patterns, not on the software program’s outdated and clunky lines. Read Muted Grid Lines: Small Details, Big Difference to explore before/after remakes in more detail and watch Removing Tick Marks and Grid Lines for a how-to lesson in Excel. Customize your color palette. Next, I swap my software program’s random color scheme for a customized palette. As a consultant, I’m typically following my client’s branding. I scroll through the organization’s website, look at their logo, and skim publicly-available reports that were created with the aid of a graphic designer. In my former role as an internal evaluator, I would match my chart colors to my organization’s own logo and branding. When I create graphs through my role with AEA’s Data Visualization and Reporting Topical Interest Group, I match AEA’s exact shade of burgundy—RGB code 149:8:4—rather than sloppily choosing any old shade of red. Editing color codes is simple. Newer versions of Excel on both PCs and Macs have built-in eyedropper tools. If you’re using an older version of Excel, follow the Uganda Evaluation Capacity Development Project’s step-by-step instructions for using a free tool called Instant Eyedropper. Write a descriptive title and subtitle. Today’s viewers want and deserve brevity, everyday language, and text that describes something about the actual finding—so that even the quickest report-skimmers will walk away having digested and retained the report’s contents. Bonus points: Select an important word or two from the title and make that word stand out ("chocolate" is in bold text and matches the graph’s dark brown color scheme). Then, add a one- or two-sentence subtitle ("Cookie dough was second most popular flavor"). Rad Resource: Want to master these skills and more? I’m leading a pre-conference workshop at the Eastern Evaluation Research Society’s conference in April 2015. Bring your laptop so we can build these charts and more from scratch. See you there! The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Lyn Paleo on Graphic-Based Reports and Graphics for Color-Impaired Readers DVR TIG Week: Ann K. Emery and Stephanie Evergreen on the Data Visualization Checklist Manny Straehle on the ICA Data Visualization Competition

Kirk Knestis, here, CEO of Hezel Associates and EERS Communication Chair. My colleagues are very good at researching education innovations and evaluating programs. We serve as RESEARCH PARTNER or EXTERNAL EVALUATOR for research and development (or R&D) projects across a variety of programs, many funded by agencies like the National Science Foundation. Lesson Learned - I emphasize the distinction between the above roles very purposefully. The first studies an innovation being developed, to inform iterative design and assess its promise of impact; the second examines the implementation and results of those R&D efforts. Both require collection and analysis of data but it’s easy to get them tangled up where they meet when planning a research project or writing a proposal. We’ve come to understand that the simplest way to keep things straight is often to work backward, first asking how best to evaluate R&D activities, then designing the research and development processes in a separate discussion. We frankly need to use more robust methods than the "panel review" evaluation approaches to which we’ve typically defaulted. Hot Tip - If one party is implementing an R&D project consistent with precepts of the Common Guidelines for Education Research and Development, and another is charged with program evaluation for that effort, make sure that everyone involved is on the same page regarding the Purpose, Justification Guidelines, and Guidelines for Evidence to be Produced detailed in the Guidelines for the type of research being designed. These attributes define the quality of the R&D so should guide evaluation of implementation and results. Hot Tip - Equally, you may largely ignore the Guidelines for External Feedback Plans in that document. That list of possible structures for organizing evaluation activities provides little useful guidance beyond raising the possibility of peer review. Unfortunately, that bears practically only on published reports of Efficacy, Effectiveness, and Scale-up Research (per the Guidelines), so it’s not an answer for many—perhaps most—R&D projects. Rad Resource - At least, I hope it’s rad. Hezel Associates has developed two conceptual models for evaluating education R&D projects—one adapted from ideas shared by Means and Harris at the 2013 AERA conference, and a second created from scratch by Hezel Associates. The latter is tightly aligned to purpose, justification, and evidence guidelines for Design and Development Research (Type #3), where most of our partners’ projects are situated. The document linked-to above is tailored to NSF Advanced Technological Education audiences, but the frameworks should be broadly applicable. If interested, take a look. If they’re not rad or if you have ideas to make them more rad, please share them. Better yet, come to the 2015 EERS conference we can talk more in person. The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Kirk Knestis on How Evaluators can Help Clients with Proposals in an Innovation Research and Development (R&D) Paradigm Leigh M. Tolley on Student Involvement in Local Affiliates Kirk Knestis on Innovation Research and Development (R&D) vs. Program Evaluation

Hello, my name is Susan Jenkins and I am Treasurer of EERS. I have evaluated several Federal Tribal Grants Programs and have learned to redesign assessment, analysis, strategies, and solutions to be appropriate for tribal governments and/or tribal programs. While American Indians and Alaska Natives are citizens of the United States, they also maintain separate and distinct citizenship, cultural values, traditions, beliefs, and identity which provide for modes of thought and communication that may differ from those of other groups. Lesson Learned: You do not have to be an expert. Learning what you can about the group you will be working with and being humble goes a long way. Lesson Learned: Allocate enough time. Tribal traditions often require that tribal leaders deliberate extensively and consider the long-term consequences of their decisions. Lesson Learned: Tribal members may speak English as a second language and some concepts are not easily translated. Being sensitive and seeking clarification in a patient and respectful manner can bridge gaps in cross-cultural communication. Hot Tip: Some ways to demonstrate respect include: Be willing to admit limited knowledge of tribal culture, and inviting tribal members to educate you about specific cultural protocols. When in doubt about something, ask respectfully for guidance. Understand that certain objects, such as feathers and beadwork may be sacred, and should not be touched or discussed. Listen and observe more than you speak and be comfortable with silences or long pauses in conversation. In tribal communities, any interruption is considered highly disrespectful, and may undermine your credibility. Understand that Native Americans may convey truths or difficult messages through humor or by telling stories. Pointing your finger is interpreted as rude behavior in many tribes. Respect personal space and do not take photographs without permission. Rad Resource: On a recommendation from the head of my agency’s Tribal Grants Program, I took the training: "Working Effectively with Tribal Governments" which provided basic skills and knowledge for working more effectively with tribal governments. I increased my understanding and awareness of tribal issues and concerns, and important legal, historical and cultural factors that should inform work with Tribal programs. Rad Resource: Medicine Wheel Evaluation Framework. This guide introduces the ‘Medicine Wheel’, outlining its history and uses, and shows how it can be used as an evaluation framework. I used this guide to develop a graphic showing proposed individual-level outcomes of the Federal Tribal Grant Program. Rad Resource: A list of citations obtained from public sources and recommended by National Indian Education Association (NIEA) staff and partners. Over 25 citations/abstracts and, where available, links to full-text are provided. The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.   Related posts: EERS Week: Lacy Fabian on an Age Old Evaluation Challenge Leigh M. Tolley on Student Involvement in Local Affiliates EERS Week: Jill Feldman and John Kelley on the Eleanor Chelimsky Forum as a Learning Opportunity

Hello! I’m Tony Fujs, Director of Evaluation at the Latin American Youth Center, a DC based non-profit organization. When asked about the role of my department, I often respond that it aims to be "the GPS unit of the organization", showing decision-makers whether the organization is on track or not toward achieving its goals. I like the GPS analogy because it is simple and easy to understand. It also provides an interesting framework to think about performance measurement systems, and how to improve them. Nonprofits Performance measurement systems Where we are:                                                                                                                                                                Where we want to be: Image credit: Biblioteca de la Facultad de Derecho y Ciencias del Trabajo Image credit: Tony Fujs                     Let’s look at a few concrete examples to illustrate this point: Lessons Learned: On data collection. Recognize that data collection is always a burden: Collect only what is needed. Eliminate manual data entry whenever possible; if not possible make the user interface as intuitive as possible. On data processing. Automate, automate, automate: Internal evaluators generally work with different instances of the same data sets, therefore data cleaning and other analytical tasks can easily be automated: Use programming tools like Excel macros or R. Modern databases can also be customized to automatically "catch" data entry errors. Hot Tip: Want to learn more about efficient data processing? I’ll be running a workshop on data management at the next EERS conference. On providing actionable information Make sure the information generated by the performance measurement system is useful and understandable for the end user. Make evaluation results hard to ignore: For instance, they could be displayed on a giant TV screen in the hall of the organization building, so nobody can enter the building without seeing them. Simple is beautiful Building a culture of data is often cited as a critical step in generating buy-in toward performance measurement systems. It is a critical step indeed, but partly because performance measurement systems are often perceived as complex and cumbersome by the end-user. Drivers adopted the GPS because it is useful and easy to use, not because they developed a culture of data. Building useful, simple, and intuitive performance measurement systems can also be a powerful and sustainable strategy to generate buy-in. The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: GOVT Week: Ted Kniker on Performance Measurement and Evaluation GOVT Week: David Bernstein on Top 10 Indicators of Performance Measurement Quality EERS Week: Jill Feldman and John Kelley on the Eleanor Chelimsky Forum as a Learning Opportunity