Program Planning and Evaluation
last updated August 31, 2003


Program planning is similar to strategic planning, but on a smaller scale. This kind of planning, broadly described, involves

 

  • creating a written description of the problem that the program will address;
  • identifying the strategies the program will use to will address the problem;
  • describing the individual activities that make up the strategies;
  • allocating these activities to individuals;
  • setting a realistic timeline for each activity;
  • establishing monitoring procedures to determine whether the project is proceeding as planned or whether revisions are necessary; and
  • outlining a plan for evaluating the program periodically and/or at its conclusion.

Program evaluation, as noted in MAP's Basic Guide to Program Evaluation, should focus on achieving utility, relevance and practicality. Evaluation, the process of collecting information in order to make decisions about a program, can help an organization determine whether it is doing what it believes it is doing, improve the delivery of services, and identify what, if any, impact the program is having on its customers or clients. The evaluation process should be designed to collect the information that is most needed by the organization's management.

Any evaluation consists of three steps—planning, collecting data, analyzing the data. Planning the evaluation begins with a careful definition of the goals to be achieved. Questions to be asked in designing an evaluation include:

1. What is the purpose of the evaluation? What decisions will need to be made after the evaluation is complete?

2. Who is the audience for this information—the board, the staff, clients?

3. What information is needed to make this decision and to inform the audience?

4. What are the sources of that information—employees, clients?

5. How can that information be collected most reasonably—interview, survey?

6. When is the information needed?

7. What resources are available to collect the necessary information?

From MAP, Basic Guide to Program Evaluation.

Data collection is the second step. Data collection methods can include short surveys, document review, or in-depth interviews and focus groups. As noted in the Basic Guide to Program Evaluation, each method has advantages and disadvantages in terms of time and expense required and the quality and quantity of data collected.

Analysis of the data is the final step in the evaluation process. It is critical that the data be organized and analyzed in terms of the initial goals identified as the purpose of the evaluation. It is also important to retain the evaluation conclusions. These conclusions might be useful in understanding subsequent changes or developments in the program.

MAP additionally provides a Checklist for Program Evaluation Planning that summarizes each of these steps. Innovation Network offers a free workstation of online evaluation and planning tools for nonprofit program planning. Detailed resources on data collection are available through the Innovation Network's page on evaluation resources. Intermediary NGOs providing services to other NGOs may find useful information through MandE, a news service that focuses on developments in monitoring and evaluation methods for social development projects. MAP also provides links to overviews of other kinds of evaluations.

MAP's Basic Guide to Program Evaluation provides a useful description of different kinds of evaluations: (1) goal-based evaluations (is the program achieving its goals?), (2) process-based evaluations (how does the program do what it does?), and (3) outcome-based evaluations (is the program having the desired impact?). This guide also offers a summary of when each type of evaluation may be useful, and questions that can be used to design each of these processes. The Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources and In Search of Outcomes by the Grantmanship Center, also provide good descriptions of the purpose of and how to design an outcome-based evaluation. Most programs rely on both a process evaluation method and either of the two product evaluation methods (goal-based or outcome-based).

The National Science Foundation's User-Friendly Handbook for Mixed Method Evaluations and User-Friendly Handbook for Project Evaluation, as well as W.K. Kellogg Foundation's Foundation Evaluation Handbook (PDF, 116 pages), all offer in-depth discussions of evaluation planning together with sample plans.

Additional program management resources, including planning and project scope checklists, are available through allPM. MAP's Basic Guidelines for Successful Planning Process contains a thorough explanation of the planning process. Gerard M. Blair's Planning a Project provides a more humorous commentary on the "dos" and "don'ts" of program planning.