Building evidence

Choosing an external evaluator

Oftentimes we find ourselves with the possibility of having to hire a person or team of evaluators outside of the organization. This situation opens the door to new experiences and it would be helpful to have someone who already has experience working in your community. To help you strategize we offer the following section.

When we talk about an external evaluator we refer to a person outside of the organization, who for the most part, has not had any previous contact with the program. Generally speaking an organization may decide to hire an external evaluator, if:

  • It is a requirement imposed by the funder (foundation, government agency, etc.);
  • The organization chooses to hire someone to carry out the task of evaluation in support of their work;
  • The organization would like to collaborate with an evaluator on specific aspects of their evaluation.

Before hiring someone to carry out the work or collaborate with your staff on evaluation activities, it is important to assess their credentials for the project. Aside from requesting general information regarding the external evaluator’s work experience, it is important to know not only what she/he has participated in, but also how she/he has carried out the work, especially with community-based and culturally specific organizations.

Here are some questions that you can use to help guide you when you are reviewing an evaluator’s resume (or curriculum vitae).

  • To what extent does the professional education of the evaluator qualify her/him to carry out this job?
  • To what extent does the work experience of the evaluator qualify her/him to carry out this job?
  • What prior experience does she/he have conducting similar projects and/or with similar organizations?

Here are possible interview questions:

  • What do you see as your role in an evaluation project?
  • How would you describe your approach to evaluation?
  • How do you engage organizations like ours in analyzing information?
  • In the past, how have you incorporated community feedback into your work?
  • How do you share evaluation results with CBOs and the communities that they represent?
  • What challenges have you encountered working with other organizations like ours? How did you overcome these challenges?

In negotiating a contract, it is important to discuss who will be the owner and have control of the information. You should also discuss usage rights in meetings or presentations outside the organization. The final agreement should be in writing and included in the contract.

Likewise, from the very beginning clearly identify and establish who will be in charge of reviewing the information or reports, both the drafts as well as the final copy. Be sure to establish a strategy to terminate the contract if the external evaluator shows signs of absenteeism or failure to carry out the duties and expectations established in the negotiation of the contract.

Red flags to which you should pay great attention throughout the process (in reviewing their proposal, during the interview, during contract negotiation, and on the job) of working with an external evaluator include:

  1. The evaluator wants to employ methods of gathering information that are inappropriate for the evaluation and do not coincide with the beliefs and/or cultural practices of the community.
  2. The evaluator plans to gather, organize, analyze, interpret and present the information without consulting the work team, the organization and the community throughout the process.
  3. The evaluator does not outline a plan in the proposal or contract detailing how they will engage community feedback throughout the evaluation process. The evaluator is unwilling to modify the proposal to incorporate community feedback opportunities.
  4. The evaluator does not engage the organization, the participants or the community in developing or reviewing the final report.
  5. The evaluator shares, presents or distributes information with other entities without consulting the organization and without the organizations’ explicit approval.  

These scenarios should not be taken lightly or ignored as they can impact your work as well as the trust, credibility and reputation that your organization has in the community.

Another important aspect to consider in evaluation work is the kind of working relationship you will have with the funder throughout this project. We suggest that from the beginning you share with the funder your expectations concerning the work to be carried out by the external evaluator as it pertains to the work in your organization and with the community.

It is equally important to share the organization’s perspective, values/norms, and ethical expectations pertaining to evaluation.

  • Do not hesitate to request information and demonstrate your interest in learning and understanding the means, the requirements and the mechanisms that the funder utilizes when hiring or selecting an external evaluator.
  • Pay attention to the power dynamics and interactions between your organization, the evaluator and the funder. You may have to advocate for your organization if you notice that your perspective is not being included to the extent that you expect it to.
  • Discuss and consider within the organization and with the community all the possible consequences that might arise if your organization sets limits, decides not to carry out activities, or denies access to information that the evaluator or funder requests from your organization.