Call Us MN: 24/7 bilingual crisis line (651) 772-1611
Building evidence
What do I need?
There are many ways to collect information about your program and each approach has its strengths and weaknesses. All of the data you collect forms part of the information you can utilize for your evaluation.
Types of information
As a general rule, evaluation information is organized into two groups: numbers (quantitative) and words (qualitative).
When we talk about numbers (quantitative data), we mean information that helps us describe what exists and happens in terms of time, quantity, frequency. For example:
The average age of participants
The number of people receiving services per week
The number of sessions participants attended, etc.
When we talk about words (qualitative data), we mean information that helps us capture a deep understanding of participant’s experience in the program or their personal stories. This can include information about the why, the how, and the consequences of participating in the program. For example:
Opinions about the program
Descriptions of experience within the program
Descriptions of impact on their personal stories, etc.
What are the sources of information?
Sources of information can come from different areas to inform your evaluation, including memos, reports, mail, attendance lists at events, financial reports, donation reports, workshops and presentation reports, census data, police records, blogs, newspaper articles, radio interviews, journal articles, social media, and research published by other organizations or universities.
Places where you can access information outside your organization include: local libraries, universities, government offices, research institutes, mass media (newspapers, radio), and social media (e.g., Facebook, Twitter, and Instagram).
We suggest you create your own list of information sources since this will help you organize your work and keep track of the information that you might have and/or need for your evaluation. Utilize this worksheet to help you get started.
What tools can I utilize to collection information [data]?
There are a variety of tools that can be used to collect information. We believe that the best tool is always the one that corresponds to the circumstances in which it will be used. In other words, based on the realities of the organization, the resources available, the type of program, the participants, and the community. Consider the resources you have both in terms of personnel and funding, this will help you determine if you can use several tools—simultaneously or during different phases of the program. Using several tools allows you to obtain rich information from different perspectives.
The following table will help you understand each tool, its purpose and the possible situations we do not recommend using the particular tool. This will help you find the best “match” for your program and evaluation needs.
When can I collect information?
Deciding when to collect information can appear to be a challenge, if we think of it as a new task. You may not have considered this, but much of your everyday work includes collecting information. For example, when people come to the office in need of services, they usually complete a form with their personal information. When collecting information about your program, the only differences are the goals and timing of your activities.
Imagine you want to collect information to evaluate a new program, in its initial phase. You think you do not have all the necessary information to decide what the main activity will be in order for the program to work. One strategy could be to organize a focus group to brainstorm ideas about the program’s main activities. Another scenario could be that a program ended a couple of months ago, but you lack sufficient information to allow you to identify the program’s strengths and weaknesses. In this case asking former participants to fill out a survey could provide the extra information you need.
The most important point is to have a clear idea of the evaluation goals or the reason you want to collect evidence. This will also help you decide when is the right time to collect information.
Graph 1. Opportunities to Collect Information.
Other Considerations in Collecting Information/Building Evidence
What are the possible costs involved in conducting evaluation?
It is important to view evaluation activities in the same way you view any other job your organization performs, it will require a budget. As with any budget you will need to include potential and unexpected expenses.
Determine the essential components of your program and the ones you can substitute. Consider collaborating with other organizations or agencies to carry out the work, as a way to save money or reduce costs. Listed below are a few basic elements you should include in your budget:
Personnel: consider looking within your organization to determine whether or not you have staff that have the skills or experience to help you with this task.
Consultant: perhaps you do not have the personnel or their experience is limited such that you will need to hire someone outside the organization. This activity must not be taken lightly because it could end up with outcomes you did not hope for. To learn more about this task we offer you a complete section on Choosing an External Evaluator, which we believe will be very useful in helping you find the right evaluator for your evaluation needs.
Transportation costs: you must consider that you might have to visit several places to obtain information or that your program activities take place at different locations.
Printed materials: the program as well as the work of building evidence will require producing and printing documents, be they surveys, interviews, focus groups guides, etc. Take this cost into consideration in addition to the cost of obtaining information from places like libraries, government agencies whether in or out of state.
Office supplies and electronic equipment: in this category include the computer, any software you might need to get the job done. And currently with the use of video cameras for Skype or other programs for video conferences, audio, graphic designs, etc.
Communications: these include traditional mailing costs, professional newsletters and electronic mailings, internet, and local, national and/or international telephone calls.
Consider in detail all the activities and materials required to efficiently carry out this task. Prepare your budget considering the best and worst case scenarios. The key is to pay attention to all the details and plan for any scenario.
How can you conduct evaluation on limited funds?
Consider personnel and resources available to evaluate your program and re-allocate resources so that someone is working on evaluation activities.
Consider your current processes and how you can enhance these activities to gather evaluation information.
Consider partnering with graduate students/universities to get free or low-cost evaluation support.
Consider hiring an external evaluator and cost-sharing fees across multiple organizations.
Do not take an “all or nothing approach”…even starting with one evaluation question and then building evaluation activities over time can be useful.
Consider using free/low-cost tools that can help you gather and analyze information easily and quickly (e.g., SurveyMonkey, etc.)
TIP: Build evaluation activities into grant submissions.
