plans for assessment discussion
Janet Schofield (schof@vms.cis.pitt.edu)
Wed, 19 Mar 1997 11:12:50 +0000
Well, it's Wednesday morning and as I promised in my first message to
you I would like to suggest a plan for our discussions now that ther
has been some time for feedback and additions. The set of questions we
have to work with includes the original four I sent to you and a
number of questions/issue areas suggested by Mavis Green. Let me
suggest that we focus our first week of discussion (from today through
next Wedensday) around three issues. My current plan is for the second
week's issues to also be a combination of her questions and mine.
However, rather than laying them out now I think it is best to wait
and see if our discussions this week raise new questions that seem
even more compelling.
Mavis raised the issue of what evaluation goals should be in this
area and since it makes sense to start with goals in planning
evaluations it seems like it makes sense to put this question on the
first week's docket. I would like to suggest that we also discuss the
first two questions I posed to you earlier this week.
So our three discussion questions for this week are:
1. What should be the primary goals in setting up assessment and
evaluation in this area?
2. What are the major issues facing evaluators on projects in this
area of technology and education that are unique to work in this area.
or at least that are more common, more important, etc in this area
than in many others?
3. What are some ways of productively handling the issues referred to
in question 2 above?
Let me help to get the ball rolling by making a couple of comments
related to items 1 and 2.
Regarding item 1. I would like to argue strongly that we need to try
to evaluate "technology in use"- that is to say that when one is
dealing with technolgoy and education projects it is crucial to think
about how the technology is actually used and how it effects the
classroom context. It could well be that in some cases the indirect
impact of technology use through its impact on other aspects of
classroom functioning (e.g. it might free the teacher for more
individual discussion with students) is more important than its direct
impact. To look only at the direct impact and to ignore its impact on
important aspects of the classroom environment may well be misleading.
Regardiong item 2: One of the issues that has arisen in the project in
which I am involved (Common Knowledge:Pittsburgh- a project designed
to bring Internet access to a large number of schools in Pittsburgh in
order to stimulate its productive use by teachers and students) is
what I have called in some earlier writing the "moving target" issue.
It is very hard to plan any kind of summative evaluation (e.g. an
evaluation designed to get an overall sense of what was accomplished)
for several reasons. First, the technology and software is changing
very rapidly and in ways that influence classroom use. When we started
this project Mosaic was not even around and neither was its successor
Netscape. Thus the kinds of uses people envisioned were quite
different from those that have emerged in many cases and the new
software often led to a reorientation in the projects teachers
undertook so that they included very different activities and were
even focussed on different goals. Thus certain "baseline" data we
gathered became useless because with new activities and goals
different baseline data was needed. Of course, not only is technology
changing repidly but as teachers gain more experience with even the
same technology they come to use it in different ways. This is
especially true when many of those involved start with a relatively
low level of knowledge and experience relating to the kind of
technology they are employing as well as little experience in actually
using it effectivley in the classroom.
OK, that's my two cents worth. Please contribute the next nickle!
(Dimes, quarters or even gold nuggets of ideas are all welcome)
Janet Schofield