Evaluation and Improvement

Concern 1. Evaluation diverts resources away from the program and therefore harms participants. This is a common concern in most programs.

  • Addressing the concern: Because evaluation helps determine what does and does not work in a program, it actually helps program participants. Without an evaluation, there is little or no evidence that services actually work.

Concern 2. Evaluation increases the burden for program staff and takes their eyes off the main purpose of the program. Program staff are often responsible for collecting evaluation information because they are most familiar with and have the most contact with program participants.

  • Addressing the concern: Despite this potential for increased burden, staff can benefit greatly from evaluation because it provides information that can help them improve their work with participants, learn more about program and participant needs, and validate their successes.
  • Addressing the concern: The burden can be decreased somewhat by incorporating evaluation activities into ongoing program activities.

Concern 3. Evaluation is too complicated. Program managers often reject the idea of conducting an evaluation because they don’t know how to do it or whom to ask for help.

  • Addressing the concern: Although the technical aspects of evaluation can be complex, the evaluation process itself simply systematizes what most program managers already do—that is, figuring out whether the program’s objectives are being met, which aspects of the program work, and which ones are not effective.
  • Addressing the concern: Understanding this general process will help program leaders and staff be full partners in the evaluation, even when outside evaluators help with the technical aspects.

Concern 4. Evaluation may produce negative results and lead to information that will make the program look bad or lose funding.

  • Addressing the concern: An evaluation may reveal problems in accomplishing the work of the program as well as successes. It is important to understand that both types of information are significant. The discovery of problems should not be viewed as evidence of program failure but rather as an opportunity to learn and improve the program. Information about problems and successes not only helps the program being evaluated but also helps other programs learn and improve.
  • Addressing the concern: When evaluation results are used for accountability purposes, it is especially important that the evaluation be carefully planned and executed.

Concern 5. Evaluation is just another form of exposure monitoring. Program leaders and staff can view program evaluation as a way for funders to monitor programs to find out whether staff are doing what they are supposed to be doing.

  • Addressing the concern: The purpose of the evaluation needs to be decided and articulated from the beginning, so decisionmakers, funders, and program staff are on the same page about why it is being done. There can be an overlap of information collected throughout regular program implementation and the evaluation. However, defining the purpose of the evaluation can help differentiate processes and answer questions.[2]

[2] Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. (2010). The program manager’s guide to evaluation (2nd ed.). Washington, DC: Author.