Summary: EPA Dialogue Day 10

[Katherine Carlitz <kcarlitz+@pitt.edu>]

Public Involvement in EPA Decisions

Summary: July 20, 2001

Dialogue Day 10 Agenda: Evaluation

Dialogue moderator Bob Carlitz and host Larry Teller from 
EPA's Office of Communications and Government Relations 
welcomed participants to the final day of this Dialogue, 
and introduced today's topic: Evaluation.  Participants 
were invited to focus on four areas: 
  * Increasing the transparency of decision-making
  * Criteria for policy effectiveness
  * Criteria for public involvement success
  * Lessons for EPA from this Dialogue.

The Environmental Protection Agency would like to remind 
you that your messages during the National Dialogue on 
Public Involvement in EPA Decisions were not a formal part 
of the comment process on the draft Public Involvement 
Policy.  The Agency would like to receive your comments on 
the draft policy before the comment period closes on July 
31st.  Information on how to submit your comments on the 
draft Policy to EPA can be found at
  http://www.info-ren.org/network-democracy/epa-pip/comment/comment.shtml

For the evaluation of this National Dialogue, Resources for 
the Future has constructed an Evaluation Survey that all 
participants are invited to complete.  You will find it on 
the Web site at
  http://www.info-ren.org/network-democracy/epa-pip/join/epasurvey.shtml
Survey results will be posted on this Web site and the 
Resources for the Future (RFF) website http://www.rff.org .

Participants' comments clustered in the following three areas.

INCREASE TRANSPARENCY OF DECISION-MAKING

** Several participants, in different ways, called on EPA 
to clearly describe its decision-making process, and state 
specifically the points at which public input could make a 
difference.  In all actual decisions, EPA should note early 
in the document how the Agency had reacted to public input.  
EPA should do this even when the public input and the 
ultimate decision don't agree.  Host Larry Teller supported 
this, noting that such a policy is most likely to prevail 
if EPA middle and senior managers support it.  The use of 
neutral facilitators as record-keepers was again suggested.

**One participant called for making the EPA web page easier 
to use, with hot links to information instead of the 
present system of print links to the Federal Register 
(often experienced as unavailable and unreadable).

CRITERIA FOR POLICY EFFECTIVENESS AND PUBLIC INVOLVEMENT SUCCESS

**A posting from John V. Stone described two possible 
models for evaluation: "outcome" evaluation, in which 
success is defined in terms of goals clearly articulated at 
the outset; and "formative" evaluation, where feedback is 
continuous and goals are reshaped appropriately.  (A URL 
with details is given.)  Many participants agreed on the 
need for continuous feedback and continuous attention to 
feedback.  There seemed to be implicit agreement that EPA 
needs to pay attention to feedback early in any decision-
making process, and at least fine-tune goals accordingly.   

**One participant suggested that public involvement success 
be measured by the level of public comment, and suggested a 
variety of ways to gauge the level.  Other participants 
objected to this approach, saying that members of the 
public who were completely satisfied with EPA's 
participation process would simply feel no need to weigh in 
(a detailed example from New York State was offered).  A 
bell-curve model offered by David James suggested that 
people in complete disagreement with EPA's objectives would 
be equally unlikely to weigh in.

**Several participants discussed the potential of market-
research tools like customer-satisfaction surveys and focus 
groups.  EPA was cautioned not to use these tools without 
informed guidance; it is easy to make mistakes by 
generalizing survey results inappropriately.  However, the 
participant who offered this caution had a number of 
specific suggestions for using surveys to capture the input 
of those who cannot participate otherwise because of 
situational, cultural, or financial barriers.  

**Panelist Eric Marsh outlined features of the draft Public 
Involvement Policy and noted that EPA does plan to develop 
customer-satisfaction survey forms.  
 
**Analyzing press coverage of EPA actions, and a conference 
for face-to-face discussion of specific instances of public 
discussion, were both suggested.  

**One participant suggested using an April 1998 EPA report 
titled "Final Guidance for Incorporating Environmental 
Justice Concerns in EPA's NEPA Compliance Analysis" as a 
tool for evaluation.  This report is said to offer useful 
guidance on how to conduct environmental assessments.

LESSONS FROM DIALOGUE FOR EPA

**Panelist Eric Marsh urged participants to fill out the 
survey evaluating the Dialogue (see URL above).  Panelist 
Caron Chess cautioned against lumping Agency participants' 
comments in with everyone else's, since Agency personnel 
will probably have a different perspective from that of the 
non-Agency public.

**One participant would like to have the final day's 
discussion be about linkages between all of the Dialogue 
agenda topics.  He worries that since the topics have been 
treated as discrete, EPA will fail to look for fruitful 
connections.  Another participant said that each topic 
should have been given more than one day.

**Several participants (and one panelist) warned against 
making the evaluation process too much of an end in itself.   
One participant cautioned against "group-think," where a 
group values its internal harmony over the achievement of 
an outcome.  

**Showing that EPA is indeed listening, Lisa Kahn reported 
that she and Pat Bonner will be analyzing the Dialogue, and 
have been tabulating all of the "worst practices" messages.  
She will be posting her list (130 pp. at time of writing) 
to the Dialogue website. 

**There was a lively exchange about the value of tolerance, 
with some participants concerned that too much tolerance 
will let polluters off the hook.  Others answered that only 
some degree of empathetic listening to a polluter would 
lead to effective measures.  While civility ultimately won 
in this exchange, EPA was called upon to champion public 
health and environmental protection, no matter how 
controversial its necessary actions.

CORRECTION:  Fred Stoss's database of subnational agencies 
and health sites is compiled by and must be accessed 
directly from him, not from the National Council for 
Science and the Environment, as stated in the Summary of 
Day 9.  (See his message on Day 9.)

Each day's summary is intended to capture the essence of 
the conversation.  While this summary contains the 
highlights of participants' comments relating to today's 
topics, more comprehensive information may be found in the 
individual postings.  This and all daily summaries are 
available from the Agenda page of the website.

  http://www.info-ren.org/network-democracy/epa-pip/join/agenda.shtml

The dialogue for today's discussion is available at 

  http://www.info-ren.org/network-democracy/epa-pip/archive/date-i1.html

Katherine Carlitz
Reporter