Formative Evaluation

[John V. Stone <jvstone@umd.umich.edu;jvstone@glerl.noaa.gov;;;;;;;>]

I think it will be important for EPA to distinguish between "outcome" evaluation and "formative" evaluation. Today's guiding questions are heavily weighted toward the "outcome" side of the ledger -- that is, retrospective measures of the degree to which specific particapatory objectives were met. That, of course, is very useful in and of itself; however, many evaluation research organizations (see, e.g., http://www.feraonline.com ) place greater emphasis on "formative" evaluation -- that is, prospective and collaborative stakeholder/evaluator feedback during the project or process being evaluated. As in outcome evaluation, formative evaluation begins with clearly defined objectives. But unlike outcome evaluations that focus on how closely these objectives were met once the project/process was completed, formative evaluation seeks continual feedback so that activities may be altered or adapted should they be producing interim outcomes that diverge from the original objectives. In this way, by the end of the project/process, outcomes will be largely consistent with stated objectives because they were continually adapted to meet project/process realities.

To that end, EPA should not view evaluation as something that is done at the end of a cycle (as reflected in its placement at the end of this on-line dialogue), but rather as something that should be strategically planned at the start of a cycle as participatory objectives and benchmarks are being mutually developed in consultation with stakeholder populations. Of course, this begs the question "who are the stakeholder populations and how are they best defined and consulted?" I will continue to advocate ethnography for that purpose, with formative evaluation of the public participation process throughout a project lifecycle.

Cheers,

jvs