We are internal evaluators Cath Kane, of Weill Cornell Clinical & Translational Science Center, and Jan Hogle, at the University of Wisconsin-Madison Institute for Clinical & Translational Research. The 62 Clinical and Translational Science Award (CTSA) hubs — along with community-based and industry partners — collaborate to advance translational research from scientific discoveries to improved patient care by providing academic homes, funding, services, resources, tools, and training. Lesson Learned: How do CTSA hubs define "translational?" Translation is the process of turning observations from the laboratory, clinic, and/or community into interventions that improve individual and public health. Translational science focuses on understanding the principles underlying each step of that process. Hot Tip: Ask "What are we evaluating?" Internal evaluators determine whether programs are efficiently managed, effective in meeting objectives, and ultimately impacting the process and quality of biomedical research — involving multiple variables in complex systems. Case studies use qualitative data to complement quantitative data on research and training productivity. Analysis of multiple case studies identifies factors that facilitate or impede successful translation. Retrospective or prospective? Retrospective analyses use an abundance of data to deeply study known successes. Prospective case studies can identify factors that influence translation in real time. Questions might include: What does ‘successful translation’ mean? Which operational process markers are most important? What are the ideal duration metrics? How does collaboration and the individual CTSA hub move translation along over time? How can we better support the translational process? The retrospective case study below tells a story of key operational markers in the development of Gleevec, used to treat leukemia. Public data such as publications, FDA approval and patenting events, and mortality data were overlaid with interview data from key informants. Lessons learned: An individual face-to-face interview is indispensable for obtaining an insider’s view of the research process. Record the interview and take notes. Often the research details are unfamiliar to the interviewer. Hand-written notes alone may not capture the interview accurately. Ask investigators for their interpretation of "translation." People have different ideas about the meaning of translation. What are the key moments or markers in translation? Case studies can serve many needs from short summary "stories" for public relations and newsletters, to more rigorous approaches paired with quantitative data for decision making by internal and external stakeholders. Perhaps the most important lesson learned is the value CTSA evaluation teams bring to their hubs, not only for long term objectives, but also in the shorter term through contributions on a daily basis to programming adjustments and course corrections using a mixed methods approach to understanding complex change. The American Evaluation Association is celebrating Translational Research Evaluation (TRE) TIG week. All posts this week are contributed by members of the TRE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Prevention research can have a huge impact on population health, but how do we evaluate the impact and translate the research into products for public health practitioners? We have been tackling that question at the Prevention Research Centers (PRC) Program at the Centers for Disease Control and Prevention (CDC). I’m Erin Lebow-Skelley and I work for the Evaluation and Translation Team that evaluates the impact of the PRCs, and I want to share our approach with you. The PRC Program directs a national network of 26 academic research centers, each at either a school of public health or a medical school that has a preventive medicine residency program (See figure, below). The centers are committed to conducting prevention research and are leaders in translating research results into policy and public health practice. All PRCs share a common goal of addressing behaviors and environmental factors that affect chronic diseases (e.g. cancer, heart disease, and diabetes), injury, infectious disease, mental health, and global health. Each center conducts at least one core research project; translates and disseminates research results; provides training, technical assistance, and evaluation services to its community partners; and conducts projects funded by other sources (CDC, HHS, and others). As a result, the PRC network conducts hundreds of projects each year. The Evaluation and Translation Team is tasked with the challenge of demonstrating the impact of this heterogeneous group of research centers. We have spent the last two years developing the evaluation plan for the current 2014-2019 PRC funding cycle, while engaging various stakeholders throughout the process. We started by developing the evaluation purpose, questions, and indicators, and now have a complete and piloted data collection system and qualitative interview guides. We plan to annually collect quantitative data from each PRC that reflects their centers’ inputs (e.g., faculty and staff), activities (e.g., technical assistance, research activities), outputs (e.g., research and practice tools, peer reviewed publications), and impacts (e.g., number of people reached) using a web based data collection system. Having a cohesive system that collects information allows us to link center activities to outputs and impacts (e.g., showing what partners were involved in X project, that contributed to Y impact), which provides a comprehensive understanding of elements that contribute to center and network impact. Hot Tip: Always start with program logic (after engaging your stakeholders!). No matter how complex the program, determining the overarching program logic will help guide the development of your evaluation indicators and provide a comprehensive picture of how the program is working. Hot Tip: Consider providing end-users an electronic means of systematically providing feedback within the information system itself pertaining to data entry problems, subject matter questions, and suggestions for improvement. he American Evaluation Association is celebrating Translational Research Evaluation (TRE) TIG week. All posts this week are contributed by members of the TRE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello! Our names are Natalie Wilkins and Brandon Nesbit and we are both evaluators at the Centers for Disease Control and Prevention (CDC), in the National Center for Injury Prevention and Control (NCIPC). One of the projects we provide evaluation support for is the Injury Control Research Centers (ICRCs) program, funded through NCIPC. This has provided us with a number of important lessons learned around evaluating multi-site research center programs that are engaging in translational research and outreach. There are 10 ICRCs across the country, funded to conduct innovative research on the prevention of injury and violence.  These institutions serve as training centers for the next generation of injury and violence prevention researchers and act as information centers on injury and violence prevention for the public.  ICRCs are also pioneering innovative approaches to the translation of research to practice. They conduct translational research studies and engage in a variety of outreach activities to translate research on evidence-based injury and violence prevention strategies into practice settings. For example, one of the ICRCs works with partners to assess their capacity for using research findings in their work, and then provides tailored technical assistance based on each partners’ specific needs to ensure research is translated into practice.  In addition to these "research to practice" activities, some ICRCs are also employing a "practice to research" approach to their translational research, leveraging their outreach activities and partnerships in the field to inform their research priorities. As evaluators of this comprehensive, multi-site research center program, one of our challenges was to show the impact of the ICRCs’ translational research and outreach activities on bridging the gap between research and practice. To this end, CDC and the ICRCs developed a set of indicators to capture information on impact (e.g. studies, partnerships, outreach activities, development of research and practice tools, etc.). We display data on these indicators through Tableau, software that allows users to analyze, visualize, and share data in an interactive way. Hot Tip: Visually presenting evaluation data through interactive dashboards allows stakeholders to glean their own insights while still ensuring key messages are communicated.  Tableau enables us to showcase the approach and impact of each of these unique research centers, while also providing the option of presenting a "bird’s eye view" of the impact of the entire ICRC program as a whole. Lesson Learned:  Translational research and outreach can take many forms. Engage your stakeholders in the evaluation process early so you can ensure they have a clear understanding of the kinds of information you are looking for. Rad Resource: For more information on how evaluators have used Tableau, check out the AEA365 archives- http://aea365.org/blog/?s=tableau&submit=Go he American Evaluation Association is celebrating Translational Research Evaluation (TRE) TIG week. All posts this week are contributed by members of the TRE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello.  My name is Sally Thigpen and I work in the National Center for Injury Prevention and Control (NCIPC) at the Centers for Disease Control and Prevention (CDC).  As an evaluator, I often find myself encouraging scientists and stakeholders to think about evaluation from the very beginning of any study or project.  I do whatever I can to be included as early as possible so I can help build evaluation into every step of the process.  The value of inclusion is equally true for our practice partners.  They need to be included as early as possible in our scientific thinking because they are vital to the translation of research to practice.  Practitioners speak to the relevance and utility of the science and the value it has to current programmatic or policy efforts.  In today’s budgetary realities, understanding these practical aspects of uptake helps assure limited dollars have the maximum impact. The Division of Violence Prevention within NCIPC developed the Rapid Synthesis and Translation Process (RSTP) to systematize this communication loop between the research and the field of practice. This six-step process (in the graphic below) can help users facilitate the negotiation between the science and practical application. Hot Tips: Before engaging with a group of practitioners, do a gut check with the scientists of record.  Ask questions about what they see as the most valuable aspect of the study for practical application.  What is their biggest apprehension about how the science might be misinterpreted and used in ways it was not intended?  These answers not only help to focus the translation efforts, but also offer a little insight as you begin working with a selected group of practitioners. Work with the same group of practitioners from the beginning to the end of the translation process.  Begin this relationship with similar questions from above.  How do they anticipate using the science?  What is the significant contribution to the field?  What is least valuable? As the translational product moves through development, keep checking in with the group of practitioners and scientists.  Practitioners can guide you on relevance and balancing science with action.  The scientists can guide you in making sure you are keeping scientific integrity along the way. Lesson Learned: You’re not just a communicator/evaluator/researcher - you’re a negotiator.  In the role of translator, you are often negotiating between the details of pure science and the brevity of the practical world.  This is a critical role and takes finesse. Rad Resource:  My colleagues and I published an article in a July 2012 special issue of the American Journal of Community Psychology reviewing RSTP’s usefulness in the field, Moving Knowledge into Action: Developing the Rapid Synthesis and Translation Process Within the Interactive Systems Framework. he American Evaluation Association is celebrating Translational Research Evaluation (TRE) TIG week. All posts this week are contributed by members of the TRE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello, my name is Jayne Corso and I am the community manager for American Evaluation Association and the voice behind the AEA Facebook page. If you manage a company Facebook page, you might have noticed a drop off of "likes" recently. Facebook has begun removing memorialized and voluntarily deactivated accounts from Pages’ like counts. This change ensures that data on Facebook is consistent and up-to-date—but could mean a drop for your analytics. Although some Pages might lose "likes," they could also gain a more accurate way to track their followers. I have compiled a few tips for tracking your analytics and gaining more visibility for your page. Rad Resource: Take advantage of Facebook "Insights" Facebook offers Page Insights after at least 30 people have liked your Page. Use this tool to understand how people are engaging with your Page. With this tool, you can see your Page’s growth, learn which posts have the most engagement, find demographic information about your audience, and identify when your audience is using Facebook.  This data is available for free and can easily be customizable for time frame and downloaded to excel. Rad Resource:  Use Google Analytics to track effectiveness Tracking your analytics through Google allows you to see how many people are coming to your site from social networks, understand the website pages they are most interested in, and gain a better understanding for how your audience is engaging with your web content.  To find this information, enter your Google analytics account and go to "Acquisitions". From here you can look at the performance of your social networks as an overview or look more specifically at referrals, activity, and user flow. All of this data allows you to gage the effectiveness of your social campaigns. Hot Tips: Increase your Facebook likes Finally here are a few simple tips for increasing the likes on your Facebook Page—hopefully you can make up for any followers you lost when Facebook made their changes. Add the Facebook icon to your website, so visitors know you have a presence on the social network (Place the icon high on the website page, near your navigation) Add the Facebook icon to your email communication or blog to reiterate your presence on Facebook to your subscribers Cross promote your Facebook page on your other social media sites. You may have followers on Twitter that have not liked your Facebook page or didn’t know you had a Page Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hello, I’m Stephen Maack, Owner and Lead Consultant of REAP Change Consultants, founded October, 2002. The Fourth of July is this week. Many become independent evaluation consultants because of the lure of freedom - freedom to choose projects, what you do, how you do it; freedom from the politics of organizations, bad bosses, and organizational hierarchies; freedom to set your own hours and work in your pajamas. It all sounds great. It’s all true, too - partly. Yes, you can choose which projects to take on but as Max A. Eggert and Elaine van der Zeil put it "Clients tend to be more rigorous and constraining than managers," and "Every client wants the right solution to his or her need without taking exceptional risks. Thus you have to have controlled creativity…."[1] Yes, work in your pajamas - but you also need to meet deadlines and juggle time for yourself, your family, loved ones, and friends. Try shutting down your computer and walking away at 5 p.m. when the final report is due tomorrow and you must get it done now because you spent time with your spouse earlier! Yes, you might be free of your own organizational politics and can walk away from those of other organizations once you finish a project. However, for good client relations, pay attention to the politics of your clients’ organizations! So what to do? Take hints from young America. Once the American Revolution resulted in independence, leaders of the young nation set about forming thirteen disparate states into the United States of America and created a new constitution. As you go independent you’ll figure out your own rules for operation, what you will do and how you’ll go about doing it. The young nation worked at developing allies and establishing markets for its goods and services. You’ll have to do that. Find business partners, learn from fellow consultants, explore and start marketing. Each of the states had others with whom to work. You do, too - AEA and the IC TIG. Go ahead. Declare independence. Just remember that bravery, responsibility, and hard work come with freedom. Like each of those who financed the revolution and founded this country, you are putting your fortune, fame and the welfare of your family on the line. Good luck! Hot Tips: Join the Independent Consulting TIG for community and advice. Take AEA training workshops. Rad Resources: Gail Barrington’s Consulting Start-up and Management: A Guide for Evaluators and Applied Researchers. Elaine Biech’a The Business of Consulting: The Basics and Beyond. Second Edition. [1] Max A. Eggert and Elaine van der Zeil (2011) The Perfect Consultant: All You Need to Get It Right First Time. The Perfect Series. Arrow Business Books, Random House Kindle edition.   The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I’m Bernadette Wright, founder of Meaningful Evidence, LLC. As an entrepreneur, I noticed a proliferation of books, papers, courses, and other sources espousing conflicting ideas of how to succeed in business and entrepreneurship. For evaluators who are building independent consulting businesses, this makes finding the best ideas for our business difficult. In my and Dr. Steven E. Wallis’s recent research on entrepreneurship, we found a frequently mentioned problem was a lack of agreement on what entrepreneurship means. Some sources portray entrepreneur as those who start ventures, generally small businesses. Others present entrepreneurship as about being innovative, pursuing opportunity, or filling a need. Several studies discussed the need to integrate theories of entrepreneurship across disciplines and across levels of analysis (e.g. individual, organizational, and market levels). Studies have also recommended coordinating entrepreneurship theories across research, education, and practice. A 2006 U.S. Association for Small Business and Entrepreneurship (USASBE) white paper titled Are We Teaching Small Business Management to Entrepreneurs and Entrepreneurship to Small Business Managers? recommended making teaching more consistent with the literature. However, the fragmentation of theories makes choosing the best theories to teach and apply difficult. Integrative Propositional Analysis (IPA) is an emerging method to analyze and integrate theories across studies, strategic plans, and other documents. IPA draws on a long research stream related to systems thinking. This scholarship indicates we can improve a model by understanding its structure and making it more inter-connected. We can use IPA combined with empirical data and feedback from stakeholders to assess plans before we implement them. IPA is a six-step process. For details, see our white paper on "The Science of Conceptual Systems." Benefits for business leaders include: Lets you choose theories that are more likely to work in the real world (as compared to in the academic world) Gives you more "structured" knowledge, which research indicates increases managers’ chances of success Supports communication, collaboration, and shared understanding Avoids unanticipated consequences Rad Resource: "A Revolutionary Method to Advance Entrepreneurship Theories" by myself and Steven Wallis. In review, draft available upon request. Discusses our study on using IPA to integrate and improve entrepreneurship theories. Rad Resource: Casson, M. Entrepreneurship and the theory of the firm. Journal of Economic Behavior & Organization, 2005. This study creates an integrated theory combining insights from management and economics. Rad Resource: "Strategic Knowledge Mapping: The Co-Creation of Useful Knowledge," by Steven Wallis and myself. Developments in Business Simulation and Experiential Learning, 2015. Introduces a gamified version of IPA. Where do you find business/entrepreneurship information to consider for your business? How might your firm benefit from using theories that are measurably better than your competitors’? The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Susan Wolfe and I am a Senior Consultant at CNM Connect where I provide evaluation and other capacity building services to non-profit organizations. Two months ago I was the owner of Susan Wolfe and Associates, LLC, my own consulting firm; however, after five successful years I chose to close my doors and become an employee. People have asked why I made this choice to give up my independence. The main reason is that although I LOVED the work I was doing, I did not so much care for the business side of independent consulting. When I was offered the opportunity to join the non-profit CNM Connect as a consultant I easily accepted it without once looking back. Lesson Learned: Even if I am successful with something, if I don’t enjoy it then it’s OK to make a change when an opportunity that better fits my interests and needs. When I started as an independent consultant it fit well with my interests and needs, but things changed over time. When I was no longer enjoying my work, I allowed myself to be open to other possibilities, and because of this a more suitable situation presented itself. Throughout the past five years, I had a love-hate relationship with working alone at home. I liked the convenience of not commuting and having to dress up for work every day. I also liked being able to work with one of my dogs in my lap. On the other hand, after some time I started to feel the absence of co-workers. I missed belonging to an organization with other people but I knew I did not want to grow my consulting business. Hot Tip: If you like having colleagues that you work with daily while at the same time would like to be an independent consultant, consider forming a company or collective. Working at home alone can be isolating. Hot Tip: Networking with other evaluators, locally, statewide, and nationally, and teaming up for collaborative proposals and projects is one way to reduce your sense of isolation. I regularly scheduled lunch with colleagues or community partners, and ultimately teamed up with another local evaluator to form an informal local evaluator’s group that gathers bi-monthly for happy hours. Rad Resource: If you are thinking of transitioning from independent consulting to working for an organization, scan the AEA job listings to see what is available. It’s a great way to look around and see if there is something that would better fit your current needs and interests. The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I’m Gail Vallance Barrington and I’m an independent consultant. It’s hard to teach novice clients about logic models. This year, instead of presenting a technical monologue, I came up with a story to capture their attention. It’s called My Rich Uncle and it goes like this: Icebreaker: You suddenly discover that you have a rich and eccentric uncle who lives far away and he wants to give you $30,000 to buy a car. Working in small groups, decide what your uncle’s name is and where he lives. To get the money, you will need to prove to him why you need the car. Use this worksheet and have fun coming up with the answers. [N.B. Do not provide the labels below until the exercise is over.] Activity: List three compelling reasons why you need the car and the difference it will make in your life. [Program Purpose, Long-term Outcomes] Your uncle wants to know that you are a good planner. What will you need to have in place before you get your car? List three or four important things you need. [Inputs] He wants to know how you will use your car on a regular basis. What will your main uses be? [Activities] Your uncle is pretty picky, and of course, he lives very far away. He wants to be sure that you actually spent the money on a car and that you actually use it. Every six months, he wants some concrete proof that you have been using it as planned. What evidence can you put in a brown paper envelope and mail to him? [Outputs] Finally, after three years, you suddenly hear from your uncle. He is coming for a visit! He wants you to show him what changes the car has made to your life. Can you start planning for it now? [Short- and Intermediate-Outcomes] Reflection: Once the groups have reported their answers, reflect on such topics as the group interaction, breakthrough thinking, general conclusions and the fact that everyone has just developed a logic model. Next Step: Launch immediately into developing the logic model for their program. Having set the stage in this relaxed and fun way, it is easy to make connections with their own context. Lessons Learned: I have used this exercise with novice clients, students and webinar participants. Every time the typical problem of differentiating between Outputs and Outcomes seems to melt away. Further, the accessible nature of the story provides clarity about a funder’s need for accountability. Next time you teach logic models, why don’t you try it? Let me know how it goes. The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi there! I am Ana García-Femenía, (@anagfemenia), PhD in Sociology and Political Sciences and independent evaluation consultant. I live in Spain and conduct evaluations for various international donors in different countries. Have you ever been in the middle of an interview in an evaluation and felt some specific emotion with no reason for it? What happens when you stop and ask yourself: Why do I feel this way? Over the years I have understood how important it is to take care of my emotional well-being, in order to do my job as evaluator. But one could say this holds true for any profession, doesn’t it? Well, yes it does. But it holds especially true for our profession where we as evaluators are confronted with constant decision-making requiring sound judgement and a special degree of calm, cool and collectedness. There is a list of situations and times when we need to exercise our best judgement not only during the evaluation process itself, but with clients and stakeholders in order to make the right decisions, to say ‘no’ when we need to, and to maintain our integrity. There are no purely rational decisions. Whether negative or positive, our emotions always let us know when we are or not on the right track. Lesson Learned: Make emotional care part of your self-care. Two Hot Tips to increase your emotional care: Meditating: for 15 minutes once a day. Meditation provides peace and serenity and opens your mind and your heart. It helps to improve your relationship with your emotions. It also provides a good preparation for full attention and deep listening. Dancing: A lot of stress and tension are relieved through dancing. Dancing helps to free our emotions. And it is a great way to socialize, which is very valuable for those of us who work alone! Rad Resources: A wonderful tool for the initiated in meditation and the uninitiated alike. It provides guided meditation ranging from 5 minutes to one hour. Online resources for dancing are huge. It also will depend on the music you like! Here are three links on the benefits of dancing for your emotional health: http://www.adta.org/ http://www.sciencedirect.com/science/article/pii/S0197455612001104 http://awakeningtothedance.com/2012/02/01/dancing-to-free-our-emotions/ And finally, for those interested, a beautiful video by one of the best world tango couples to get inspired! The American Evaluation Association is celebrating IC TIG Week with our colleagues in the Independent Consulting Topical Interest Group. The contributions all this week to aea365 come from our IC TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.