NSF Program Effectiveness Review

Common Knowledge: Pittsburgh

Creating an Urban Research Laboratory

Project Report - March 20, 1997

Common Knowledge: Pittsburgh (CK:P) was invited by the National Science Foundation to participate in its March 20, 1997 program effectiveness review (PER). As directed, this project report will focus on three areas of concentration - effectiveness of the CK:P project at achieving its goals, catalytic effects of the project and migration of CK:P activities from research to implementation. This report will document our successes, failures and lessons learned. It will describe the benefits of NSF funding for testbeds that are collaborative in nature and centered in urban school districts.

Phase I - Common Knowledge: Pittsburgh (Funding through AAT, 1993, 1994)

CK:P is a collaboration involving the Pittsburgh Public Schools (PPS), the Pittsburgh Supercomputing Center and the University of Pittsburgh. The primary audience for grant activities are the students and staff of the school district. Initial funding for Common Knowledge: Pittsburgh came through NSF's AAT program. In addition CK:P received funding from a number of other sources. Two local foundations - the Buhl Foundation and the Vira J. Heinz Endowments - provided matching funds during the initial AAT phase of the project. During this time period, CK:P also received a Department of Commerce TIAAP grant - Bridging the Urban Landscape. This grant enlarged the CK:P collaboration, implemented the CK:P model in a community center, built new online resources, and explored technology options.

The goals of the Common Knowledge: Pittsburgh project were listed in its original proposal to NSF's AAT program:

Teaching Environment. The network shall enhance the teaching environment for all teachers in the Pittsburgh Public Schools by expanding the personal contacts and information resources available to these teachers.

Curriculum Development. The network shall be available to serve the development of curricular activities of the Pittsburgh Public Schools in all subject areas and at all grade levels.

Access to Information. The network shall provide information services not currently available to the Pittsburgh school system.

School Restructuring. The network shall serve as a tool in the restructuring efforts of the Pittsburgh Public Schools.

A National Model. The network and its associated curricular activities shall serve as a national model for the implementation of wide area networks in the K-12 environment.

These goals are aligned with NSF's focus on turning research into practice.

The first three goals - Teaching Environment, Curriculum Development and Access to Information - pertain to research into the learning environment and how it is influenced by the availability of network technology. This aspect of our research corresponds to the original two year AAT grant. This grant provided CK:P collaborators with an opportunity to develop technological tools, curricular models and support mechanisms for project implementation.

The goal of enhancing the teaching environment was accomplished in a number of ways. CK:P technical staff created a distributed, scalable and supportable technical infrastructure. This included on-site network servers (maintained remotely), client software packages, and central support services. Local Area Networks (LAN) were interoperable providing IP to the desktop. PPS users had access to Internet services including email, Usenet news, gopher, World Wide Web, WAIS, ftp. CK:P provided access to Internet services in three ways. First, teachers at CK:P sites had access via their school LAN. By the end of the AAT grant - 1995 - this was available at 11 schools. Second, all PPS staff had home-access through the University of Pittsburgh and Carnegie-Mellon modem pools. Finally, CK:P provided electronic, phone and software support. This was a significant aspect of the project. Any PPS staff member, from custodian to superintendent, could obtain a free Internet account, access that account through a free modem pool and obtain whatever support necessary to setup hardware, install software and learn how to navigate network clients. Access and support created an environment that lent itself to professional growth, empowerment and autonomy. Success with respect to this goal was measured by growth in the user population (over both the AAT and NIE phases of the project):

• 3000 staff (approximately 60%) have Internet accounts,
• 1500 students (at 14 CK:P sites) have Internet accounts,
• 2/3 of PPS students and staff have access to the Internet via their school libraries.

Also, as a result of the CK:P project, a substantial number of staff has purchased and is using technology for the first time. CK:P support staff continues to spend a large part of its

time helping individuals go online from their homes and classrooms.

The teaching environment has changed. Faculty use electronic mail to collaborate with their colleagues. There is rapid growth of local (CK:P and PPS) newsgroups, mailing lists and WWW pages. Users are becoming more sophisticated in their use of the network - asking how to attach files to email messages, obtain software on the net, and deal with issues of ethics and acceptable use. The language and culture of PPS staff has changed as a result of networking experience.

CK:P took a very different approach to the goal of curriculum development. Our belief is that curriculum development is a process where teachers confront issues pertaining to content, school culture, organizational barriers, personal beliefs, ownership and accountability. Since this process is dependent upon individual needs, developmental issues, and school specific initiatives, CK:P decided that it would not create curricular applications or books of lessons for other teachers to use. We felt that this would best be done by those people who implement curriculum - the teachers themselves.

Since CK:P was using a staged implementation (4 sites in 1993, 7 sites in 1994, 3 sites in 1995, 6 sites in 1996 and 9 sites in 1997) a selection process had to be developed. Typical district policy would have a school director choose the schools based on need, familiarity, principal or politics. CK:P developed a Request for Proposal (RFP) process that was a departure from existing district procedure. Through the RFP process (/projects/ckp/publications/articles/rfp/cover.html), interested schools had to submit proposals explaining how they would use the technology to influence content, curriculum, professional growth and student achievement. A review committee of 24 stakeholders - parents, teachers, administrators, school board members, and representatives from local foundations - chose the sites based on quality of proposal and probability of success. CK:P staff facilitated the review process, but did not participate as reviewers.

The RFP process allows groups of teachers to define their network activity to meet their professional and classroom needs. The RFP process provides a unique opportunity in a public school setting for entrepreneurship and ownership. It fosters professional growth. We judged success with respect to the goal of curriculum development through interest in the RFP, effect on the curriculum and maturity of proposals over time. Data collected indicates that this goal was met:

• 33 out of 89 schools participated in the 1^st year - 1994 - of the RFP,
• 9 out of 14 middle schools participated in the 2^nd year - 1995 - of the RFP,
• 9 out of 11 high schools participated in the 3^rd year - 1996 - of the RFP, and
• 43 out of 69 schools participated in the 4^th year - 1997 - of the RFP.

This shows exceptional interest across the district. A review of existing CK:P sites and their curriculum applications of the network indicate the RFP has had the desired result. Although this information will be provided in depth by our evaluators after the project's conclusion, online information provided by the schools (/projects/ckp/curriculum/curriculum.html) indicates that the network has had a profound effect on curricula and the classroom. A few stand outs include:

• Science classes at East Hill Elementary participating in the "Winter Solstice" project - an online data collection which taught geography, longitude and latitude, seasons and planetary motion;
• Students and staff at Westinghouse High School accessing online experts via Usenet newsgroups. Students obtaining expertise from chemists and environmentalists regarding research in acid mine drainage and the effects of acid rain on ghost shrimp;
• Students at Schenley High School using the Internet Relay Chat to converse in real time with students in Germany and Spain;
• Students at Liberty Elementary publishing their poetry and images on the WWW and receiving feedback from local writers;
• Students at Frick Middle School debating with students in Norway on common articles read in national newspapers;
• Liberty, East Hills, and Fulton Elementary are using the Network to participate in an online problem solving forum. They use resources from the Mathematics Forum at Swarthmore College as well as local and internationals mathematicians to provide problem samples.

By providing PPS teachers with the tools to access resources and the necessary support, they demonstrated their creativity and perseverance at developing innovative curricula.

The goal of access to information is somewhat obvious. Network access in school libraries and classrooms provides students and teachers with resources that they never imagined. CK:P's experience taught us two interesting points with respect to access. First, the librarian is a key player in the use of network technologies in schools. We found that use of network resources demands a new set of skills - the ability to conduct research. Neither students nor staff have a great deal of experience with keyword searching, triangulation of data, and issues of authenticity. However, the school librarian has a great deal of skill and expertise in this area. Every CK:P site has a cluster of computers in the library. Also, most non-CK:P sites in the district have a single access point via modem in their library. The librarians have shown themselves to be invaluable human resources in accessing and interpreting the information available on the Internet.

The second point with regard to access pertains to a broadening of the concept of resources. Most initial attempts to use the Internet focus on database resources. Staff especially became enamored with online WWW and gopher resources. However, student behavior indicates a desire to access other humans who are online. Students want to talk to other students, to adults who are expert in areas of interest, and to people in the community. Developmentally, children and teenagers are more interested in people than databases. The Internet provides a community of human resources that are always looking to help one another. Use of the Internet raises issues of appropriate use and supervision. Instances of inappropriate use of the network have been minimal due in large part to CK:P's acceptable use policy and the vigilance of the faculty. Increasing access to information and human resources is a goal that has been successfully met.

Phase II - Creating an Urban Research Laboratory (Funding through NIE 1995, 1996, 1997)

The second phase of Common Knowledge: Pittsburgh began with our proposal to NIE - Creating an Urban Research Laboratory (URL). This proposal asked NSF to fund an effort to continue the research focus of the project, translate the research into practice in the district, and provide resources for others interested in building programs similar to Common Knowledge. In its review of the Creating an Urban Research Laboratory proposal, NSF indicated that the proposal did not go far enough towards migrating past efforts into the district. NSF forced this issue in a very powerful way when it moved the project from AAT to NIE, and demanded that the principal investigators, project administration, technical support and budget be located in the school district, not the university. This forced the district to take ownership of the financial and operational aspects of the program. The revised proposal funded by NSF was consistent with these changes.

Moving control of the project to the PPS forced the district to consider what institutionalization of the project might mean. This concept of institutionalization corresponds to the fourth goal of the original grant - School Restructuring. Since being funded, CK:P and PPS have grappled with these concepts - what is institutionalization, what is restructuring? On one level institutionalization pertains to funding. Matching funds for this second phase of the project were not from local foundations, but from the Pittsburgh Public Schools. The district, which continues to struggle with funding its technology plan, has paid for all of the wiring and computers in the three year, NIE phase of the project. Certainly contribution of funds is one form of institutionalization.

Another aspect of institutionalization pertains to the technical infrastructure of the project. In this area, there are many indications that institutionalization is occurring.

• PPS staff use of the university modem pools became so overwhelming that the universities asked that we find a way to transition users to some other modem pool. CK:P felt that at-home modem pool access was necessary for project success. Providing at home access for staff allowed individuals the opportunity to develop their interest and skills on their own time. Through funding from NSF and a great deal of work by both CK:P and PPS technical staff, a PPS modem pool is now in place. This modem pool takes advantage of existing district infrastructure - an 80 line modem pool used for administrative purposes. The new PPS modem pool has 24 lines for day time access and a total of 80 lines for evening and weekend access. Thousands of PPS teachers use this modem pool to access the Internet.
• The CK:P server design has been adopted by the district. Pentium based PCs running BSDi Unix as their operating system are used at school sites. They are maintained and updated from a central site. Although this migration is not complete, there is agreement that PPS technical staff will perform this function by project completion.
• An even clearer indicator of the effect of this project on the district is what occurs at schools that are not part of the CK:P project. Seven non-CK:P schools found funding to implement their own networks. They have installed LANs that use the CK:P design, connect to the central PPS site using a CK:P developed ISDN topology, have site servers that are clones of CK:P servers, and use CK:P client software to access the Internet.

Another form of institutionalization is creating the language and tools that the institution needs and uses in its day-to-day operations. An example of this is that the PPS Divisions of Computer Services, Facilities and Student Information consider the CK:P model to be the appropriate LAN/MAN/WAN design for the district. They use both the technical language and tools developed by CK:P in implementing network technologies throughout the district.

The timeline for migration (/projects/ckp/process/mschedule.html) and the issues raised in the migration process (http://common.peabody.pps.pgh.pa.us/~wertheim/migrate2.htm) are directly effected by the district's strategic plan and available human resources. Migration of technical infrastructure has been slower than expected. Migration of educational support has not occurred at all. This can be attributed to the lack of personnel and funding within the district. To clarify this statement, a digression to organizational issues is relevant.

CK:P started with a collaborative staffing model:

- technical staff from the Pittsburgh Supercomputing Center,

- educational staff (on special assignment) from PPS, and

- project management and evaluation from the University of Pittsburgh.

It is important to note that no PPS staff was (or is) in a full time, district funded position to support project activities. Rather, technical staff would have to be identified and either hired or redirected within PPS and trained by CK:P. The same would have to occur for educational staff and project management.

During the AAT portion of the project, we benefitted greatly from the ability to hire talented staff to develop and deploy our model. However, this was not the case in the NIE part of the project, especially with regard to educational support personnel. CK:P now has the task of porting the project to the district and handing it over to people who were not included in a primary way from the beginning. Although the pertinent PPS staff is on advisory committees and informed of all project activities, it is not responsible and therefore reticent to put effort into the project. It is this reticence that is slowing the migration process down.

This is not an insurmountable hurdle. A number of activities may help the migration:

-PPS is contemplating a technology plan, that can provide resources for this effort.

-Years 4 and 5 of CK:P have a significant amount of money for district central infrastructure.

-PPS just went through an organizational restructuring that places clear responsibility for instructional technology under CK:P's primary PI - Dr. Bernie Manning.

-There is significant collaboration between CK:P and the PPS Division of Computer Services who will manage the technical aspects of the project.

With the project coming to a close, it is clear that the migration must be completed quickly. The educational support necessary to continue project efforts remains cloudy. The four teachers that manage CK:P for the district will have to be integrated back into the district after funding is gone. It is unclear whether they will go back to the classroom or become involved in supporting instructional technology within the district. If they are placed into support positions, the district will have to create four new positions for this effort (presently funded by NSF). With the district's present financial status, this could be a hardship.

We are getting closer to a definition of "implementation" or "institutionalization". From the perspective of language, culture and behaviors we have developed the critical mass necessary to sustain the project after funding is over. From the organizational perspective, many difficult questions remain. We hope to have these questions answered by the end of this funding cycle.

Phase III - Lessons Learned and Future Funding

The fifth goal of the original proposal - a National Model - corresponds to sharing our lessons learned with other educators. Although we have not finished our evaluation of the project, the lessons learned through the process of implementation are not difficult for us to enumerate.

Lessons Learned from a Research Perspective

• Project management should be within the organization being acted upon.

• Both educators and technical experts have a role in product development. They should work together, side-by-side, in the actual environment where the product will be used.

• It is easy to become obsessed with products. The focus must be on the process.

• The first layer of teachers or schools that pursue change are not typical or the norm; this would indicate that more support will be needed as the project proceeds.

• Involve all of the players from the very beginning.

• Reward positive efforts.

• Formative evaluation is a key element to keeping proper perspective on the process.
• Change is an evolutionary process that needs to be measured over a number of years.
• Initial plans for Internet use often shift markedly over time as teachers encounter unanticipated roadblocks and exciting new opportunities.

Lessons Learned from an Implementation Perspective

• Don't be naive, assess organizational variables realistically, be proactive.

• Consistency is key.

• Communicate the process of implementation publically to all stakeholders.

• People will judge your actions and results, not your rhetoric.

• Change is demonstrated at the classroom level.

• Change is institutionalized at the administrative level.

• Structural changes must occur:

- inservice, support and professional development needs to be ongoing and institutionalized,

- teacher empowerment includes real, informed decision making,

- time must be provided for experimentation, dialogue, continued research and planning.

These lessons learned reflect on organizational structure, management, human needs and culture.

CK:P has learned that every aspect of implementing instructional technology is charged with tension. From the moment a site is chosen as a CK:P site, an activity ensues that touches upon every aspect of the organization.

-The project has to work with the Division of Facilities to develop LAN design. This includes working with district architects, electricians, inspectors and vendors.

- The project has to work with the Division of Finance, the Division of Computer Services and Computer Repair to order, inventory, install and maintain the computers.

-The project has to work with building principals and staff to determine what computers to buy, where to place them, who will maintain local infrastructure, how staff will be trained, and how support will be provided.

-The project has to work with the school board and the Division of Strategic Planning to determine how to integrate the project into the district's overall strategic plan.

-The project has to work with the union and administration to eliminate roadblocks.

-The project has to work with parent and community groups to educate, inform and create positive PR.

-The project has to get into the classrooms to see its effects on students.

-The project has to work with district support and curriculum personnel to integrate its efforts with other curriculum, instruction and assessment efforts in the district.

We list this litany of activities to demonstrate that creating a model that can be disseminated via conferences, publications, and traditional means is not sufficient to fully understand the complexity of the process one has to participate in it.

Common Knowledge: Pittsburgh believes that in order to create a National Model and to fulfill the promise of the Urban Research Laboratory, one must engage potential schools and districts in the process of implementation. We presently have successful models in elementary, middle and high schools in the Pittsburgh Public Schools. We have successful community models, through the TIAAP grant, at the Hill House Association and its neighboring schools. We also have, to quote an administrator in the PPS, "institutionalized demand". CK:P is using this NSF program effectiveness review to suggest that we can provide a vehicle for sharing the lessons learned on a larger scale. To do this, three items must be accomplished:

The Pittsburgh Public Schools must pass its technology plan which builds on the foundation the NSF helped to create by funding Common Knowledge: Pittsburgh. A district technology plan will provide the local capital to implement instructional technology across every school in the PPS.

Outside funding must be obtained to allow the CK:P model to be expanded to community centers around the school district.

Federal funding must be obtained to continue the research started by CK:P, to continue the educational support provided by CK:P, and to open the Urban Research Laboratory in the PPS to other school districts around the country.

We envision providing internships to other school districts who want to learn about the technical, human and political infrastructure necessary to have successful technology implementation. These teams of educators and technical staff would spend 9 weeks working with CK:P staff and district personnel as they wire schools, purchase equipment, install LANs and train site personnel.

Common Knowledge has tackled the issue of implementation in a systemic fashion. It would be simplistic to believe that we could simply publish our story or lessons learned and hope that others could build on our model. In working with staff, students and administration, we have taken a constructivist model of learning and applied it to organizational change. The Urban Research Laboratory would be a testbed for others to begin their education regarding instructional technology and implementing it in a systemic manner.