Several years ago, long-time family and friends got together over the holidays for a good meal.  Grown children were back in the nest, and we enjoyed the repast reminiscing about days gone by. The conversation, however, drifted into a discussion regarding what we remember as the most upsetting or painful interaction we had with our parent/child.  For example, one son mentioned his father yelling at him across a blackjack table, "If you're not going to play RIGHT, don't play at all." What emerged was that every child had a painful...

My name is Charmagne Campbell-Patton and I am an independent evaluator. This was my first year at MESI, and I really appreciated the focus on social justice. One of the main themes that ran across most of the presentations was the power that comes with the role of evaluator and our responsibility to use that power to promote social justice. Here are just a few of the salient points I took away with me: Hot Tip: Speak truth to power. In his workshop with Nora Murphy on wicked problems in Developmental Evaluation, Michael Patton likened the evaluator to the Medieval Jester, whose role was to deliver bad news to the Monarch in a way that could be heard. To drive home the point, Michael donned a jester mask, which he wears when beginning a new evaluation assignment to lighten the mood. In addition to providing some levity to tense situations, the mask is a good reminder that evaluators have a responsibility to tell the truth, especially when no one else can or will and in particular, to those who may not want to hear it. Cool Trick: Empower others to speak for themselves Michael’s presentation certainly made an impression, but he is a white man with a PhD and 45 years of experience as an evaluator, all of which make it much easier for him to speak truth to power than for some. So how do you get people to listen if you’re not in a position of power? One option is to empower those who are traditionally marginalized to speak for themselves by providing them with data that gives them a voice. In his keynote, Eric Moore, Director of Research and Evaluation at Minneapolis Public Schools, emphasized the importance of paying attention to the theoretical frameworks that inform evaluation work because the design of an evaluation determines who has a voice and who is at the table to make meaning of the data. Hot Tip: Hold yourself accountable. When asked what he did when those in power didn’t listen to him or those he was trying to give voice, MQP shared that sometimes he had to walk away. Indeed, in his closing words, Eric Moore offered this piece of advice: Be accountable on a daily basis to yourself and why you are doing the work. Lesson Learned: Evaluation in a social justice context takes courage. Perhaps that should be added to Jean King and Laurie Stevehan’s essential competencies for evaluators? Rad Resources: Classic analyses: Hill, A. (1997) Speaking truth to power. New York: Doubleday. Hoppe, R. (1999). Policy analysis, science and politics: from ‘speaking truth to power’ to ‘making sense together.’ Science and Public Policy, 26 (3): 201-210. Spiegel, A., Watson, V., & Wilkinson, P. (1999). Speaking truth to power?. The Anthropology of Power. Wildavsky, A.B. (1979). Speaking truth to power: The art and craft of policy analysis. New Brunswick, NJ: Transaction Books. The American Evaluation Association is celebrating MESI Spring Training Week. The contributions all this week to aea365 come from evaluators who presented at or attended the Minnesota Evaluation Studies Institute Spring Training. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hey aea365ers, Sheila B. Robinson here, aea365’s Lead Curator and sometimes Saturday contributor with an update on what YOU are reading. Each year, we look at the top posts from the past year. We have to wait a bit since some posts get significant readership weeks or even months later! Lessons Learned: Here are the top posts of 2014, as measured by Feedburner: Sheila B. Robinson on Delightful Diagrams from Design Diva Duarte (April 12, 2014) Susan Kistler on Innovative Reporting Part I: The Data Diva’s Chocolate Box (April 3, 2014) Jordan Slice on LucidChart for Freeflow Chart Construction (June 8, 2014) DVR TIG Week: Ann K. Emery and Stephanie Evergreen on the Data Visualization Checklist (June 27, 2014) Elissa Schloesser How to Make Your Digital PDF Report Interactive and Accessible (January 28, 2014) Jayne Corso on Creating Images That Will Make Your Presentations Stand Out (December 6, 2014) Molly Ryan on Using Icon Array to Visualize Data (August 13, 2014) William Faulkner and João Martinho on Visual Communication with a Poster: A Quick, Dirty, and Biased Case Study (November 17, 2014) Literature Search Strategy Week: Molly Higgins on the Best Databases for Everything (November 3, 2014) Clare Strawn on Evaluating Communities of Practice (October 31, 2014) Hot Tip: Definitely some great authors and great reading in that group. And hey, I’m in there too (it was a surprise to me!). It’s clear that our readers like to learn about new resources and innovative ways to use them in evaluate work. All of those post links are still live of course, so seize the moment and catch up on anything that you may have missed! Cool Trick: Want a chance to see your name in this list a year from now? It could happen! Send me a note of interest at aea365@eval.org. I’d love to hear from you. Congratulations to our Top 10 authors and many thanks to them for their wonderful contributions. *A very special thank you to Susan Kistler, AEA’s Executive Director Emeritus for helping extract the data for this post! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

On July 8, @SHRMnextchat chatted with John Dooney (@SHRMAnalytics) about "HR's Roller Coaster Ride From the Great Recession." In case you missed it, here are all the great tweets from the chat:     [View the story "#Nextchat RECAP: HR's Rollercoaster Ride From the Great Recession" on Storify]  ...

Hi!  We are Nathan Anderson, Data Management Specialist for Mid-Dakota Education Cooperative, and Amy Engelhard, Data Steward for the state of North Dakota. We strive to help PK-12 educators move from a "data frustration" mindset to a "data utopia" mindset. More specifically, we collaborate with teachers and administrators to optimize the use of student achievement for purposes of informing individual student instruction, identifying strengths and weaknesses in a classroom, and illuminating trends and gaps across a school district. We often embed the Statewide Longitudinal Data System (SLDS) Data Use Standards and the A+ Inquiry framework into our presentations and work sessions as strategies for teaching and facilitating effective data use processes. The SLDS Data Use Standards resource introduces essential knowledge, skills, and professional behaviors. A+ Inquiry is an effective data use framework centering on the premise of awareness, which connects a wheel-like series of stages that make up a thorough inquiry cycle. Stages include absorbing the correct context; asking an essential question; accumulating, accessing, and analyzing the right data; answering the question; announcing the findings; and applying decisions and actions. Graphic adapted from "Disciplined inquiry: Using the A+ Inquiry framework as a tool for eliminating data hoarding, mindless decision-making, and other barriers to effective ESA programming," by N. C. Anderson, M. R. Brockel, and T. E. Kana, 2014, Perspectives: A Journal of Research and Opinion About Educational Service Agencies, 20(3).   Lesson Learned: Many teachers and administrators have access to seemingly endless amounts of data they don’t know how to use. When educators access data while first asking strong questions, they are better equipped to find valuable answers and put the data to effective use. Accessing data without first asking a strong question puts many educators at risk for wasting time and other resources on a purposeless data pursuit.  Frequently, this is where the "data frustration" mindset takes hold. Hot Tip: Use A+ Inquiry as a framework to examine how the SLDS Data Use Standards can look in action and to visually put yourself in the "data utopia" mindset. Hot Tip: Align your inquiry processes with the SLDS Data Use Standards and A+ Inquiry to reduce mindless decision making, data hoarding, and frustrations affiliated with using data. Rad Resource: Take a look at the SLDS Data Use Standards resource. The SLDS Data Use Standards Workgroup is in the process of creating new enhancements to the original resource. Rad Resource: A+ Inquiry was introduced in an article (scroll to No. 4) explaining how an Education Service Agency could apply the framework. We have now adapted the framework for use with student achievement data. Rad Resource: Check out these A+ Inquiry resources, including a one page handout, presentation with A+ Inquiry slides, and effective data use scenarios written through an A+ Inquiry lens. The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.  

Joe Gerstandt is one, of a group of people, you can find here in the HR/Social Media space who call Omaha, NE home.  Joe is a frequent keynote speaker on general HR topics and also in his own area of expertise on Diversity and Inclusion (D & I).  Joe is also one half of the duo called Talent Anarchy.  Under that banner, Joe works with his partner Jason Lauritsen.  Together they speak, host conference events and write books. Joe has been working the speaking circuit for more than...

We are Krista Collins and Eugenia Gwynn, experienced evaluation consultants for Early Childhood Education (ECE) programs in Atlanta, GA. Over the past decade, ECE programs have become a major funding priority, with topics such as family engagement, pre-kindergarten, school transitions, and quality rated standards receiving a lot of attention. We have worked closely with programs taking the lead to define these innovative practices, and have supported their efforts to develop, measure, and validate best practices in ECE. We recently collaborated on an evaluation of a local ECE community of practice (COP) convened to develop a family engagement framework to guide quality rated standards and practices throughout the state. To support these efforts, our evaluation focused on understanding how the COP stakeholders defined family engagement practices. With such a wide variety in how stakeholders were invested in family engagement, we had to capture this information using many different techniques. Below are a few successful strategies we employed in an effort to identify a singular definition of family engagement that summarizes the multiple perspectives of an ECE COP. Lesson Learned: Learn the Language. Reviewing the existing literature is a great place to start, but learning how your stakeholders talk about the topic and identifying the resources they use can provide valuable information about the local context. Make time to become familiar with the organizations and policies that govern how stakeholders operate. Speaking the same language will not only help stakeholders participate in the evaluation more effectively, but it will also help you understand the data and form conclusions more efficiently! Hot Tip: Just Ask! The most informative data we received came from a simple brainstorming session with parents. We invited parents to tell us what they did to be engaged in their child’s education, and what centers did to encourage their involvement. Instead of conducting a formal focus group with a prepared plan and list of questions, we facilitated a brainstorming session and had parents work together to define what family engagement in their community looks like. The best part - we were able to quickly see how family engagement practices look differently across communities! Rad Resource: The Family Involvement Network of Educators (FINE) is comprised of educators, practitioners, policymakers, and researchers dedicated to strengthening family-school-community partnerships. The FINE Newsletter, which is published monthly by the Harvard Family Research Project (HRFP), shares the newest and best family engagement research and resources from HFRP and other leaders in the field. The newsletter provides useful information about family engagement, including research reports, teaching tools, and training materials. The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hiring conditions are perhaps in their strongest stretch since the Great Recession ended six years ago. Job openings have reached a level not seen in nearly 15 years, and more workers are quitting their current positions in search of something better, thus creating churn and opportunities for other job seekers. Problems do remain: One-third of HR professionals in the manufacturing sector (33.7 percent) said they experienced increased recruiting difficulty in June, as did 27.5 percent of service-sector HR professionals, according to the Society for Human Resource Management’s Leading Indicators of National Employment (LINE) report. June marked the 16th straight month...

Greetings! We are Kelly Murphy, Program Chair of the PreK-12 TIG, and Selma Caal, Research Scientists from Child Trends, a non-profit, nonpartisan research center that works to improve the lives and prospects of children and youth. As developmental psychologists and program evaluators, we spend a lot of time thinking about how to best measure program impacts on youth and strive to develop measures that are rigorous, developmentally appropriate and feasible for program contexts. While we are pleased to see a shift from a narrow focus on academic performance and problem reduction, to a broader "whole child" perspective that encompasses outcomes, such as social-emotional learning (SEL) and "soft" skills, we realize this shift has brought new challenges for educational evaluators: Defining SEL outcomes, and distinguishing them from other positive youth development outcomes; Identifying which SEL outcomes are relevant to the population served; and Identifying reliable, valid and brief measures of SEL outcomes. Today, we’d like to share some rad resources you can use to assess SEL outcomes in your own evaluations. What is social-emotional learning? Broadly, SEL encompasses a number of skills that promote positive relationships, ethical and conscientious work, and productivity. Child Trends recently identified five key SEL skills that help students excel in school over time: self-control, persistence, mastery orientation, academic self-efficacy, and social competence. If you’re interested in learning more about SEL check out these Rad resources: The newly released Handbook of SEL Research and Practice; The Collaborative for Academic Social and Emotional Learning (CASEL) has numerous resources on defining and measuring SEL; and StriveTogether has an overview of social and emotional learning competencies and has published a report that provides a detailed review of SEL outcomes and their relationship to academic achievement.  How do you measure social-emotional learning outcomes? Rad Resources: If you’re interested in findings measures of SEL that are relevant across various developmental periods: Child Trends, in collaboration with the Tauck Foundation, has published a report on measuring elementary school students’ social and emotional skills, which includes teacher-report and student-report measures of social and emotional skills that are free to use; Additionally, Child Trends, in collaboration with the Templeton Foundation, published a book on measures of flourishing children, which includes free measures that have been tested rigorously with adolescents. These measures can also be accessed online; StriveTogether has also published reports that include a summary and a compendium of SEL measures; and Performwell, a collaborative effort initiated by Urban Institute, Child Trends, and Social Solutions is a searchable database of measures. The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello! We are Sheila A. Arens and Mariana Enriquez. Evaluators frequently propose psychometrically sound instruments or low-inference observation protocols used by trained observers for data collection. Such instruments and methods are sometimes the most appropriate to address a question of interest. However, we believe it is important for evaluators to stretch beyond traditional data collection methods when it makes sense to do so and when traditional, quantitative approaches will not be able to reach the target population, capture their experiences, or produce meaningful, reliable data.  It is important to give careful consideration to how and whether respondents will engage in data collection, and even when traditional methods may be desirable, to think about whether there are more appropriate ways of delivering the data collection request to the target population. In other words, data collection cannot be a one-size-fits-all approach. We frequently work with populations for whom completing surveys or participating in focus groups would be difficult. In fact, these data collection methods may even further disenfranchise these individuals. For instance, some may feel immense discomfort discussing any issues in a group, or may quickly identify whose lead they need to follow in the group of participants to remain in a "safe place" … yet, the evaluator may persist in using focus groups as the means of data collection. Lesson Learned: Recognizing that data collection cannot be a one-size-fits-all approach, we have sought alternative ways to engage individuals. We must be nimble and creative in our data collection approaches. And, although alternative methods may not work for everyone, how the target population is informed and engaged may make a difference in their participation. Including "participants" in the process to decide the best ways to reach them—and being humble about what we know, what works, what’s best, etc.—not only seems prudent but also seems like a culturally competent approach. Alternative data collection approaches might include approaches that are similar to focus groups but provide additional opportunities for sharing not circumscribed by a single facilitator or necessarily hampered by group dynamics, the use of photographs or images produced and annotated by participants (photovoice, for instance: http://steps-centre.org/methods/pathways-methods/vignettes/photovoice/ ), or online social networks such a Twitter or Google Hangout. A word of caution: Be creative, but be sure that your data collection method is appropriate for the evaluation questions of interest, is sensitive to participants’ needs and existing resources, and that the evaluation budget can support the additional burden of a potentially more time-consuming analytic method. Rad Resources: PhotoVoice, an app for iPhones (respondents can upload images and record their thoughts about the image) WorldCafe design principles The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.