¡Salud! and Seasons Greetings! We are Lisa Aponte-Soto and Saúl I. Maldonado, AEA GEDI alumni and co-chairs of LA RED, the Latina/o Responsive Evaluation Discourse Network. Aponte-Soto serves as National Program Deputy Director of RWJF New Connections at the OMG Center for Collaborative Learning. Maldonado is Adjunct Faculty at Santa Clara University’s School of Education and Counseling Psychology. LA RED is a network of Latina/o AEA members driven by a collective mission of transformative leadership to: (1) increase the representation, engagement, and leadership of Latina/os in evaluation; and, (2) develop professional discourse for Latina/o Responsive Evaluation (LRE) theory, methods, and practices. Today, LA RED launches an aea365 week on Latina/o Responsive Evaluation Discourse. Below are our reflections on this week’s lessons learned, rad resources and hot tips from LA RED membership: Art Hernandez, Lisa Aponte-Soto, Wanda Casillas, Leah Christina Neubauer, Josie Serrata, Martha Hernandez, Grisel Robles-Schrader, Maria Jimenez, and Andrea Guajardo. Lesson Learned on Defining LRE Practices: Acknowledge Pan-ethnicity. Latina/o evaluators represent diverse cultural backgrounds and perspectives. As a pan-ethnic term, Latina/o, has social and political benefits and boundaries. One benefit of ethnic aggregation is the allocation of resources; one boundary is the possible blurring of specific ethnic identities. We caution that Latina/os are not a homogeneous group, and self-identification as Latina/o evaluators does not qualify as expertise for all Latina/o subgroups or communities. Attend to language and culture. Evaluators learning from, with and for Latina/o communities must respect language and cultural nuances in the customization of evaluation design and data collection instruments. Learn from History. It is important to recognize that historically evaluation practices of Latina/o communities in the U.S. have marginalized and created barriers to the perception and subsequent conduct of evaluation. Engage Community. Inclusion demands a paradigm shift from evaluations of Latina/o communities to evaluations with/for Latina/o communities. This calls for prioritizing respect when collaborating with Latina/o communities and advocating for participatory methods, from co-creating instruments and reviewing data together to amplifying our avenues for sharing findings. Participate in Dialogue. Culturally responsive evaluations with/for Latina/os requires continuous dialogue that will be negotiated and renegotiated based on the contexts of identity, locality, geography, migration, country of origin, language, and much more. Apply LatCrit. Drawing on critical race theory, we need to expand to frameworks rooted in Latino-based practices like LatCrit and question the utility of prescriptive linear and sequential logic models as the primary, and often only, pathway for evaluation design. Championing LRE. To enact these practices, recruiting, retaining and training Latina/o leadership in evaluation as well as creating the conditions for a community of practice for all colleagues conducting evaluations with/for Latina/o communities serve as distinctive, yet complementary, goals. LA RED looks forward to learning from/with you. The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse Network Week. The contributions all this week to aea365 come from LA RED Network members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Cultural Competence Week: Lisa Aponte-Soto on the Benefits of Networks for Building Culturally Responsive Practices Cultural Competence Week: Lisa Aponte-Soto and Leah Christina Neubauer on Increasing the AEA Latino/a Visibility and Scholarship Cultural Competence Week: Osman Özturgut and Lisa Aponte-Soto on Sustainability of Cultural Competency in Evaluation

My name is Art Hernandez and I am a Professor and Dean at Texas A&M University Corpus Christi. I teach and practice research and evaluation, have participated in the AEA Minority Serving Institution initiative also serving as Director. As a supporting member of the newly formed LA RED network, this post focuses on Latino Cultural Competence. Lesson Learned: The usual assumption that people from particular cultural backgrounds (especially Latino/a) are by nature culturally competent to work with that population is often not well founded. Hot Tip: While it is clear that individuals’ likelihood of cultural competence is greater when those individuals share a cultural background with those who participate in an activity to be evaluated, it should not be assumed that this competence is guaranteed by virtue of that common background. This is true for multiple reasons, a few of which are presented here for consideration: Experience: Difference of experience is a great contributor to perspective; values and differences are extremely likely across generations even within cultural groups. Cultural identity: In addition, it is likely that in some situations, differences in education, economic background, etc. have an impact on cultural identity to some demonstrable degree even if there is an overlap in experience during some previous stage of life (e.g. childhood poverty, etc.). Language: Differences in language fluency and usage between the evaluator and the evaluand are likely given differences in education among other things. These differences must be examined even when evaluators are "fluent" since even among fluent speakers differences in vocabulary, slang, etc. can often be geographically based and it is clear that language is one of the main indicators of culture and often the principal source of or medium for conduct of the "data" considered in evaluation exercises. Lessons Learned: Basic principles of cultural competence: It should be clear and a matter of usual operation that the basic principles of cultural competence be applied in all situations regardless of the apparent background of the evaluator or the ability of that individual to provide an insight into the community about and from which information will be collected. Common cultural competence elements are essential: A detailed self-assessment, direct engagement with and assessment of the community to determine values, perspectives, history, and objectives. All of these are germane and of potential influence. As such, they must be considered in the crafting of data collection and interpretive, analytical mechanisms. Rad Resource: Read Del Prado et al.’s Culture, Method and Content of Self Concepts: Testing Trait, Individual-Self-Primacy and Cultural Psychology Perspectives. The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse Network Week. The contributions all this week to aea365 come from LA RED Network members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Scribing: Vidhya Shanker on Discussions Regarding the AEA Cultural Competence Statement CC Week: Jori Hall on Integrating Cultural Competence into Everyday Practice, Part 1 MSI Fellowship Week: Art Hernandez on AEA’s Minority Serving Institution (MSI) Fellowship Experience

?Saludos! We are Lisa Aponte-Soto and Wanda Casillas, AEA GEDI alumni from LA RED network. Aponte-Soto serves as National Program Deputy Director of RWJF New Connections at the OMG Center for Collaborative Learning. Casillas is a postdoctoral scholar with the National Center for Institutional Diversity at the University of Michigan. The growing Latino presence in the United States demands a pool of Latina/os to support culturally responsive consultancy efforts; however, Latina/o evaluators are disproportionately underrepresented in the field. LA RED is commitment to helping consultancies meaningfully engage in evaluation practices with a culturally responsive lens that attends to and unpacks the heterogeneity of Latina/o communities. The following highlights lessons learned from LA RED’s Evaluation 2014 Think Tank session on pathways for building a pipeline of Latina/o evaluators. Lessons Learned: Identify partners - To increase the representation of Latina/os in evaluation, it is necessary to collaborate with cross-cultural partners currently conducting Latina/o-specific evaluation, understand community needs, and learn from successes and challenges of evaluations conducted in a variety of Latina/o communities. Build evaluation capacity of Latina/o communities - LA RED should work closely with Latina/o community-based organizations tasked with conducting program evaluations to identify potential partners, evaluation practice opportunities and professional development needs for Latina/o evaluators. Create support systems - Seasoned Latina/o evaluators and cross-cultural partners are needed to serve as padrinos and madrinas to provide mentoring, coaching, and sponsorship to emerging Latina/o evaluators in communities of practice. Facilitate thought leadership - Pláticas or discussions are critical to promoting Latina/o Responsive Evaluation (LRE) discourse and developing a culture of co-learning with communities and evaluators’ practices to advance the field. Develop an outreach strategy - Outreach is instrumental to promoting the array of resources offered to support LRE. Additionally we should promote the role of evaluation in Latina/o communities with local affiliates and graduate programs in geographic areas with higher Latina/o penetration. Create opportunities for professional development - Fostering a cadre of Latina/o evaluators extends beyond the support systems offered through networks, mentorship, coaching, and sponsorships to professional development traineeships and internships. Programs like the AEA GEDI and MSI offer educational preparation on LRE practices grounded in critical race theory but only offer a limited number of seats on an annual basis. Additional opportunities need to be established. Rad Resource #1: LA RED is partnering with the Latina Researchers Network (LRN) to provide webinar trainings on LRE practices. Visit the LRN for information on the first webinar series, scheduled for February 2015. Rad Resource #2:  The Center for Latino Community Health Evaluation and Leadership Training offers fellowship opportunities for consultants conducting Latina/o health related evaluation. The Center is hosting its annual Latino Health Equity conference in April 2015. The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse Network Week. The contributions all this week to aea365 come from LA RED Network members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.   Related posts: Cultural Competence Week: Lisa Aponte-Soto on the Benefits of Networks for Building Culturally Responsive Practices LA RED Week: Lisa Aponte-Soto and Saúl I. Maldonado on Charting the Course for Latina/o Leadership in Evaluation Cultural Competence Week: Lisa Aponte-Soto and Leah Christina Neubauer on Increasing the AEA Latino/a Visibility and Scholarship

We are Dr. Maria Jimenez, Independent Evaluation Consultant in Los Angeles, CA, and Andrea Guajardo, MPH, Director of Community Health at CHRISTUS Santa Rosa Health System in San Antonio, TX. As supporting members of the newly formed LA RED network, this post focused on the consideration of critical race theory as a practical application in Latino/a Responsive Evaluation. Latino/a Responsive Evaluation often requires a participatory research approach, and in doing so, provides opportunity to conduct evaluations within the context of a critical framework. Cultural intuition is a valuable tool to mitigate race, ethnicity, and culture in community-based programs focused on Latino/a populations. Hot Tip #1: Acknowledge issues of race/racism within the evaluation context. Latinos are a heterogeneous group representing various genetic populations including indigenous African American, European, and American. Evaluators need to understand the subtle and not-so-subtle differences between these racial populations. Critical race theory as a methodological approach can be used to address issues of race in Latino/a Responsive Evaluation. Critical race theory places race at the forefront of research, use an interdisciplinary, participatory approach, and promotes a social justice agenda. Hot Tip #2: Be aware of immigration and migration trends within the population of focus in the evaluation. According to the 2012 Statistical Portrait of Hispanics in the United States from the Pew Research Center, 52.9 million people in the United States self-identified as either Hispanic or Hispanic and one other race. Of these, nearly 19 million are foreign-born with Mexico as the most common country of origin (64.2%).   Fifty-two percent of the Hispanic children in the United States are considered 2nd generation. Pew data also shows significant differences in language, news and information acquisition preferences, and political and cultural opinions based on the length of time a person has lived in the, whether a person is undocumented or is a first, second, or third generation American citizen. Hot Tip #3: Diversify your evaluation team. Build a transdisciplinary team with diverse sociological, historical, or practical perspectives. For example, team members with strong backgrounds in Urban Chicano studies will provide a different lens than that of a team member with experience rural Tejano culture. Acknowledge the intra-racial complexities of the Latino/a population and ensure that multiple viewpoints exist on your evaluation team. Additionally, ensure that members of your team various ethnic/racial backgrounds and/or have a solid understanding of the culture of the program, participants, or communities being studied. Rad Resources: Pew Research Hispanic Trends Project Parker’s Commentary: Can Critical Theories of or on Race Be Used in Evaluation Research in Education. The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse Network Week. The contributions all this week to aea365 come from LA RED Network members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: LA RED Week: Lisa Aponte-Soto and Wanda Casillas on Pathways for Building a Cadre of Latina/o Evaluators Cultural Competence Week: Lisa Aponte-Soto and Leah Christina Neubauer on Increasing the AEA Latino/a Visibility and Scholarship LA RED Week: Lisa Aponte-Soto and Saúl I. Maldonado on Charting the Course for Latina/o Leadership in Evaluation

I am Leah Christina Neubauer with DePaul University’s MPH Program. I serve as President of the Chicagoland Evaluation Association (CEA). This post highlights my paper in the LA RED network’s AEA 2014 session (#1439): Visionary Evaluation for Building Sustainable Cultural Responsive Evaluation Practices for Latino/a Communities. From my paper, Lessons from Little Village, Public Health & LatCrit, I offer three guiding questions: When conducting evaluation with and within Latino communities, does the setting matter? Should the evaluator be Latino? Should the design and methods be Latino-focused? In response to my queries, I offer LatCrit as a framework for advancing Latino-focused evaluation dialogue and scholarship. I offer key insights and resources below. Hot Tip: Definition, Aims & Values What is LatCrit?   LatCrit is a theory which considers issues of concern to Latinas/os such as immigration, language rights, bi-lingual schools, internal colonialism, sanctuary for Latin American refugees, multi-identity, and census categories for "Hispanics". Aims: Aligned with critical legal theory roots, initial aims included: The production of critical and interdisciplinary knowledge The promotion of substantive social transformation The expansion and interconnection of anti-subordination struggles, and The cultivation of community and coalition among outsider scholars. Values: These aims are couched with a commitment to: Expansive practical programming Vast community-building structures Continual engagement of one’s self-critique, and Analysis to ensure multidimensionality. Lessons Learned: Possibilities for Evaluators and Evaluation Multiple Evaluation Approaches: For evaluators, LatCrit highlights theories and models which evoke use, participation, responsiveness, culture, indigenous peoples, social justice, and transformation. Resources: In my health work, ample time and money are essential for quality LatCrit-aligned evaluation. Evaluators (or RFP writers!) should allow time for formative or developmental processes. Many of our Latino communities have untold processes, stories, and phenomena that must be told and appropriately captured. Multidimensionality: To be clear, multidimensionality is valued, understood, and shapes all evaluation processes. In practice, this includes resources to develop a nuanced understanding of key multidimensional issues such as: history, context, community, stakeholders, language, dialect, power structures, etc. Latino-Focused Evaluator Roles: Are context-sensitive, interpreters, translators, mediators, and storytellers. They are grounded in an international, contextual perspective in evaluation. They are familiar with the community’s geographic and historical background. They bear cultural and linguistic competency. They practice multidisciplinary methodology and embody responsive and power-aware evaluation practice. Rad Resources: Clayson, Castañeda, Sanchez, and Brindis’ Unequal power—changing landscapes: Negotiations between evaluation stakeholders in Latino communities. Mertens, and Wilson’s Program evaluation theory and practice: A comprehensive guide. Valdes’ Foreword: Under construction. LatCrit consciousness, community, and theory. The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse Network Week. The contributions all this week to aea365 come from LA RED Network members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: LA RED Week: Lisa Aponte-Soto and Saúl I. Maldonado on Charting the Course for Latina/o Leadership in Evaluation Cultural Competence Week: Lisa Aponte-Soto and Leah Christina Neubauer on Increasing the AEA Latino/a Visibility and Scholarship LA RED Week: Maria Jimenez and Andrea Guajardo on Practical application of Critical Race Theory in Latina/o Responsive Evaluation

Hi all!  Liz Zadnik here, aea365 Outreach Coordinator and occasional Saturday Contributor.  I wanted to share some insights and reflection I had as the result from a recent EVALTALK discussion thread.  Last month, someone posed the following request: I’m searching for a "Why Evaluate" article for parents/community members/stakeholders. An article that explains in clear and plain language why organizations evaluate (particularly schools) and evaluation’s potential benefits. Any suggestions? Rad Resources: Others were kind enough to share resources, including this slideshare deck that moves through some language and reasoning for program evaluation and assessment, book recommendations  There is also a very helpful list from PlainLanguage.gov offering possible replacements for commonly-used words.  (Even the headings - "Instead of…" and "Try…" - make the shift seems much more manageable). Lessons Learned: Making evaluation accessible and understandable requires tapping into an emotional and experiential core. Think about never actually saying "evaluate" or "evaluation."  It’s OK not to use phrases or terms if they are obstacles for engaging people in the evaluation process.  If "capturing impact," "painting a picture," "tracking progress" or any other combination of words works…use it!  It may be helpful to talk with interested or enthusiastic community members about what they think of evaluation and what it means to them.  This helps gain insight into relevant language and framing for future discussions. Have the group brainstorm potential benefits, rather than listing them for them.  Similar to engaging community members in discussion of the "how" is also asking them what they feel is the "why" of evaluation.  I have heard the most amazing and insightful responses when I have done this with organizations and community members.  Ask the group "What can we do with the information we get from this question/item/approach?" and see what happens! Evaluation is about being responsible and accountable.  For me, program evaluation and assessment is about ethical practice and stewardship of resources.  I have found community members and colleagues receptive when I frame evaluation as a way to make sure we are doing what we say we’re doing - that we are being transparent, accountable, and clear on our expectations and use of funds. We’d love to hear how others in the aea365 readership are engaging communities in accessible conversations about evaluation.  Share your tips and resources in the comments section! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Angela Fitzgerald on Engaging the Community - an Evaluator’s Perspective Michelle Baron on Building a Culture of Assessment Bloggers Series: Charles Gasper on The Evaluation Evangelist

Happy New Year! We’re Leslie Goodyear, Jennifer Jewiss, Janet Usinger, and Eric Barela, the co-leaders of the AEA Qualitative Methods TIG. All four of us are practicing evaluators with a passion for bringing the stories and experiences of evaluation stakeholders to the fore. For the past few years, as part of editing a book together, we have been exploring how qualitative inquiry and evaluation fit together and how to identify good practice in qualitative evaluation. What are criteria for quality in qualitative evaluation? As we were reviewing chapter drafts for the book, we had to come up with a way to determine the quality of the work represented by the authors. After many conference calls and email conversations, we developed a model for thinking about the elements of quality in qualitative evaluation. Lesson Learned: High quality qualitative evaluation is grounded in a cyclical and reflective process that is facilitated by the evaluator. The following five elements make up the process: The evaluator first must bring a clear sense of personal identity and professional role to the process. It’s a matter of understanding who you are, what you know, and what you need to learn. The evaluator needs to engage stakeholders and build trusting relationships from the outset and throughout the evaluation. High quality evaluation relies on sound methodology, systematically applied, that is explicitly shared with stakeholders. Conducting quality evaluation can only be accomplished by remaining "true" to the data; in other words, hearing participants as they are, not how the evaluator wants them to be. Skillful facilitation of the process by the evaluator results in learning by all involved. Lesson Learned: The elements in the model are not necessarily progressive or discrete. All the elements are at play in an evaluation and may cycle back on each other, interact with each other, or occur in a completely different order. Lesson Learned: Although we don’t explicitly call out context as an element of the model, a strong, dynamic understanding of context is critical grounding for all high quality qualitative inquiry. Thus, context is embedded in all the elements in the model. Rad Resource: More about this model and stories from our own practice of qualitative inquiry in evaluation can be found in the final chapter of our new book, Qualitative Inquiry in Evaluation: From Theory to Practice (2014, Jossey-Bass). The American Evaluation Association is celebrating Qualitative Evaluation Week. The contributions all this week to aea365 come from evaluators who do qualitative evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: QUAL Eval Week: Katrina Bledsoe on Qualitative Inquiry and Theory-driven Evaluation QUAL Eval Week: Laurie Stevahn and Jean King on Essential Competencies for Effective Qualitative Evaluators QUAL Eval Week: Norma Martinez-Rubin on Balancing Roles and Qualitative Inquiry for New, External Evaluators

Hello everyone—Laurie Stevahn (Seattle University) and Jean King (University of Minnesota) here—continuing to grapple with issues relevant to program evaluator competencies (whether essential sets exist) and usefulness (if enhanced practice results). In fact, for over a decade we have been working on a formal set of evaluator competencies, trying to answer the daunting question of what knowledge, skills, and dispositions distinguish the practice of professional program evaluators. Applying what we’ve learned to the work of "qualitative evaluators" didn’t quite make sense because an evaluator is neither qualitative nor quantitative. As we learned in Evaluation 101, methods come second, once we’ve established an evaluation’s purpose and overarching questions; methods do not guide evaluations. So a competent qualitative evaluator is first and foremost a competent evaluator. But what is a competent evaluator? Rad Resources: Reading through sets of competencies—there is a growing number of them around the world—can be a helpful form of reflection. We synthesized four competency taxonomies: Essential Competencies for Program Evaluators Competencies for Canadian Evaluation Practice International Board of Standards for Training, Performance, and Instruction (ibstpi) Evaluator Competencies Professional Competencies of Qualitative Research Consultants Lesson Learned: Thankfully, there was overlap across the domain sets. We can say with considerable confidence that a competent evaluator demonstrates competencies in five areas: Professional—acts ethically/reflectively and enhances/advances professional practice. Technical—applies appropriate methodology. Situational—considers/analyzes context successfully. Management—conducts/manages projects skillfully. Interpersonal—interacts/communicates effectively and respectfully. Lesson Learned: What distinguishes a competent qualitative evaluator? An enduring commitment to the qualitative paradigm. Qualitative evaluators understand and intentionally use the qualitative paradigm, choosing projects with questions that require answers from qualitative data. They need technical methodological expertise related to collecting, recording, and analyzing qualitative data. Hot Tip: When using qualitative methods, focus on developing a special "sixth sense" to ensure a high-quality process and outcomes for qualitative studies. This is your ability to interact skillfully with a wide range of others throughout an evaluation to produce trustworthy and meaningful results. It involves interpersonal skills on steroids. A competent qualitative evaluator has to be attuned to social situations and skillfully interact with people in authentic ways from start to finish, knowing quickly when things are tanking. Hot Tip: In the end, highly specialized sets of competencies unique to a particular evaluator role are less important than your commitment to engaging in ongoing reflection, self-assessment, and collaborative conversation about what effectiveness means in particular conditions and circumstances. Rad Resource: Stevahn, L. and King, J. A. (2014). What does it take to be an effective qualitative evaluator? Essential Competencies. In Goodyear, L., Jewiss, J., Usinger, J., & Barela, E. (Eds.), Qualitative inquiry in evaluation: From theory to practice. Jossey-Bass, pp. 139-166. The American Evaluation Association is celebrating Qualitative Evaluation Week. The contributions all this week to aea365 come from evaluators who do qualitative evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Cheryl Poth on Articulating Program Evaluation Skills Using the CES Competencies Scribing: Anne Vo on A Radically Different Approach to Evaluator Competencies Kate McKegg on Developing Evaluator Competencies

My name is Michael Quinn Patton and I am an independent evaluation consultant. That means I make my living meeting my clients’ information needs. That’s how I came to engage in Utilization-Focused Evaluation. Utilization-focused evaluation does not depend on or advocate any particular evaluation content, model, method, theory, or even use. Rather, it is a process for helping primary intended users select the most appropriate content, model, methods, theory, and uses for their particular situation. In considering the rich and varied menu of evaluation, utilization-focused evaluation can include any evaluative purpose (formative, summative, developmental), any kind of data (quantitative, qualitative, mixed), any kind of design (e.g., naturalistic, experimental) and any kind of focus (processes, outcomes, impacts, costs, and cost-benefit, among many possibilities). Utilization-focused evaluation is a process for making decisions about these issues in collaboration with an identified group of primary users focusing on their intended uses of evaluation. Hot Tip: Involve primary intended users in methods decisions. This enhances their understanding and capacity to make sense of and use findings. Because different methods involve different timelines and require different amounts of resources, for example, qualitative inquiry being especially labor-intensive because of the fieldwork involved, deliberation on and negotiation of methods decisions should be collaborative, not decided autonomously by the evaluator, despite her or his methodological expertise. Hot Tip: Present methods options. In order for primary intended users to participate in methods and design deliberations, and make an informed decision about priority evaluation questions and appropriate methods, the utilization-focused evaluation facilitator must be able to present the primary data collection options, their strengths and weaknesses, and what makes them more or less appropriate for the evaluation issues at hand. Hot Tip: Qualitative inquiry is always on the menu of options. The evaluator needs to understand and be sufficiently proficient at conducting qualitative evaluations to present it as a viable option and explain its particular niche and potential contributions for the evaluation being designed. Hot Tip: Keep up-to-date with new developments in qualitative inquiry. New directions include uses of social media for data collection and reporting findings, increased use of visual data, and many new purposeful sampling options. Rad Resources: Patton, M.Q. (2014) Qualitative inquiry in utilization-focused evaluation. In Goodyear, L., Jewiss, J., Usinger, J., & Barela, E. (Eds.), Qualitative inquiry in evaluation: From theory to practice. Jossey-Bass, pp. 25-54. Patton, M.Q. (2015) Qualitative Research and Evaluation methods, 4th Sage Publications.  Patton, M.Q. (2014) Top 10 Developments in Qualitative Evaluation for the Last Decade.           The American Evaluation Association is celebrating Qualitative Evaluation Week. The contributions all this week to aea365 come from evaluators who do qualitative evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: ROE Week: Chris L. S. Coryn and Kristin A. Hobson on "The Dependability of Campbell Collaboration, Cochrane Collaboration, and What Works Clearinghouse Research Reviews" Sheila B. Robinson on a Utilization-Focused Approach to Evaluation Learning and Last Minute Opportunity QUAL Eval Week: Leslie Goodyear, Jennifer Jewiss, Janet Usinger and Eric Barela on Qualitative Inquiry in Evaluation

Hi. I am George Grob, an evaluation consultant focusing on policy development and advocacy. During 40 years of Federal service, mostly in the Department of Health and Health and Human Services), I learned that policy makers (Members of Congress and high level executives) are very interested in evaluations. They especially like observations, real life stories, and field reports. Hot Tips: #1. Hit the pavement. Some of our more compelling evaluations were based on onsite reviews and discussions with program beneficiaries. For example, in the face of a severe shortage of foster families, state and local agencies began intensive media campaigns to recruit them. That didn’t work. We were asked to find out why not. When we interviewed foster families they told us they joined up because they were impressed with other foster parents they had met. Our report showed that this informal foster parent network could be used as a far more powerful motivating force than the advertisements. #2. Tell them something they don’t already know. Policy makers and their staff are very well read and constantly in touch with advocates and researchers. It is important to learn where their knowledge black holes are. That’s easy. They will tell you. Work on those. For example, the U.S. Surgeon General was getting worried about wine coolers. She was afraid that kids were being enticed into drinking what seemed like a harmless alcoholic drink that could be a gateway to heavier drinking. She was right. We found that convenience stores near schools were placing wine coolers in fruit juice aisles. Bottle labels obscured the alcoholic content-dark colored backgrounds with only slightly darker and similarly colored ink; small font size. We asked kids to select wine coolers from similarly bottled fruit juices. They couldn’t do it. Our report buttressed the Surgeon General’s campaign to reduce child consumption of wine coolers. #3. Answer their questions. One of my favorite stories is about the distress of policy makers who noticed that a seemingly disproportionate amount of foster care dollars were being spent on administration instead of foster care payments. They were ready to carve out what they believed was waste and give it to the kids. They asked us to look into it. Our study found that most of what was labeled as "administrative costs" was case work—family studies to determine the best placement of a child and to prepare for court hearings. This was the very heart of the foster care system. The policy community sheathed their knives and looked deeper into the foster care system. Rad Resource: Grob, G. (2014). Qualitative inquiry for policy makers. In Goodyear, L., Jewiss, J., Usinger, J., & Barela, E. (Eds.), Qualitative inquiry in evaluation: From theory to practice. Jossey-Bass, pp. 55-76. The American Evaluation Association is celebrating Qualitative Evaluation Week. The contributions all this week to aea365 come from evaluators who do qualitative evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. Related posts: Susan Kistler on Offbeat Southern California for Those at Evaluation 2011 Susan Eliot on Evaluation Stories Bloggers Series: Susan Eliot on a Qualitative Research Blog