The School Based Partnership Program in Windsor, California: Results from the Local-Level Evaluation By Maureen Buckley, Ph. D. Sonoma State University Edited by Craig D. Uchida 21st Century Solutions, Inc. Background The Town of Windsor is located 63 miles north of San Francisco and was founded in the 1880s as a quiet farming community. Encompassing 6.5 square miles, Windsor’s 2001 population of 23,718 has grown from 14,800 in 1992 when the Town was incorporated as a common law city in July 1992. Windsor is currently the fastest growing community within Sonoma County and has the fastest growing juvenile population. Created in 1992, the Windsor Police Department is a contract department with the Sonoma County Sheriff’s Department. The department has engaged in community policing since its inception in 1992. The department follows a “Problem Oriented Policing” philosophy, has a bicycle patrol component, an “adopt-a-Cop” program, a landlord/tenant conflict resolution program, a knock and talk program, neighborhood watch program, as well as several other community oriented policing components. The Windsor Police Department also has a Youth and Family Services Bureau. The Windsor Youth and Family Services Bureau was created in 1996 (then called the Windsor Juvenile Diversion Program) with the goal of ensuring that the Windsor Police Department would respond to juvenile law enforcement problems with a pro-active, nontraditional, community based orientation. WYFS exists as a partnership between the Windsor Police Department, the Windsor Unified School District, and the Sonoma State University Department of Counseling. The student population of Windsor public schools is 3,900. Approximately 72% of the students are Caucasian and 26% of the students are Hispanic, with a small percentage of Native American, Asian/Pacific Islander, and African American students. Windsor High School, the focus for this program, had approximately 1,152 students in 2001 (up from 900 the previous year). Founded in 1996, Windsor High School moved to a new site in 1999. The Windsor Unified School District partnered with the Town of Windsor and the Windsor Police Department through the creation of Windsor Youth & Family Services, and the School Resource Officer position. Overall the relationship between Windsor High School and the Windsor Police Department has been positive, and problems that have developed over time have been efficiently resolved. Implementation of Problem Solving Scanning The initiation of the grant process occurred through the Windsor Police Department, particularly the Juvenile Diversion Program. According to the original grant application, a variety of information sources contributed to the identification of the problem of “students in possession of/or under the influence of drugs and alcohol on/or around campus.” Law enforcement information played a key role and included: officer observations, routine crime analysis, notice of repeat calls for service at or near the school, recognition of repeat offender patterns, and recognition of repeat victims. Information from community and school meetings and consultations also informed the selection of the problem and included participation from students, parents, school personnel and community members. The Program Coordinator noted stakeholders from the following organizations: Windsor High School, Circuit Rider Productions, The Windsor Police Department, Vineyard Faith Ministry, Windsor Middle School, Sonoma State University, Cali Calmecac School, and the Windsor Police Department. In addition to representatives of these agencies, several individuals were also identified as stakeholders. Analysis The Analysis phase of this project included participation by school personnel, law enforcement representatives, representatives from Sonoma State University, Windsor High School Students, the Peer Helping Program, and parents. A local community based organization, Circuit Rider Productions, also participated. The analysis phase extended beyond the planned period. Reported obstacles included difficulty achieving consensus among the partners regarding how to best assess the issue of student drug and alcohol use. For example, there was a disagreement regarding the value of implementing a student survey and the appropriate content of such a survey. Reportedly, the school initially opposed the administration of a student-created survey related to drug and alcohol use. The Project Coordinator reported that concern about administering such a survey coincided with the shooting at Columbine high school. This historical factor reportedly influenced the school’s reluctance to administer the survey, given the potential for negative media attention. After protracted discussion, occurring over the course of a year, an alternative survey was designed and administered to select students at the school. Although attempts were made to distribute the survey to a representative sample of Windsor students, the survey was ultimately distributed to what would best be described as a “convenience sample.” Another reported obstacle during this phase related to the hiring of a school site coordinator for the grant. The original partnership agreement included a plan to hire an existing school employee for this position. Unfortunately, this was not possible due to union issues (primarily regarding overtime) and the difficulty in opening up a new position for hiring. Analysis data were gathered from a variety of sources including law enforcement records (arrest reports, incident reports, and juvenile citations), academic records, consultation with stakeholders, community meetings, and a survey of parents and students (student sample N= 190; parent sample N=61), as well as the California Healthy Kids Survey. The parent and student surveys provided open-ended questions related to substance use, consequences, and intervention in the town of Windsor. Unlike the California Healthy Kids Survey, these surveys did not ask questions about personal behavior but, rather, about students and parent’s perceptions of “Windsor High School Students.” In using these data sources, the partners attempted to obtain specific information regarding the crime triangle elements of offenders, victims and locations. It should be noted that, early on in the process, the partners determined that, in specific instance of student drug and alcohol use, young people may simultaneously be categorized as offender and victim. Response The Response Phase occurred during the spring semester of 2000 and throughout the 2000/2001 academic year. The responses focused on improving the handling of the incidence of the drug problem as well as enhancing the community’s ability to respond to the issue in the future. These goals were selected based on Scanning/Analysis findings. A primary concern was that 80% of juvenile recidivism related to drug or alcohol violations. In addition, student survey data indicated that many students were unaware of the consequences of illegal substance offenses. Together, the information suggested that interventions implemented at the first citation, or prior, may be optimal. As such, the partnership chose to improve the school curriculum by initiating a public safety course, with instruction coming from law enforcement officers. An on-campus School Resource Officer (SRO) introduced new educational experiences, such as driving under the influence activities. Not only was the SRO critical in enforcement and education, but also his physical presence in the environment served as a potential obstacle to illegal behavior due to the increase in perceived risk his presence instilled. The law enforcement agency’s Youth and Family Services Bureau, which works with student offenders of those at-risk for offending, sought to enhance early intervention efforts. As part of this effort, Youth and Family Services Staff collaborated with the Student Attendance Review Board, the goal of enhancing early intervention in truancy cases. Analysis data also indicated that many students did not feel comfortable discussing drug or alcohol issues with adults in the community. For this reason, enhancements were made to the Peer Helper program at Windsor High School. Because student survey data suggested that students may use drugs or alcohol as a coping strategy for dealing with personal stresses and problems, it was anticipated that an improved Peer Helping program would potentially offer a more health-promoting outlet for these concerns. Because various analysis data sources suggested that substance use occurred during unstructured and unsupervised time, efforts were made to increase the amount of after-school activities offered through the Teen Center. Finally, because analysis data pointed to a notable gap in parent-student communication related to drug and alcohol use, attempts were made to hold parent meetings. The Project Coordinator reported that these meeting were poorly attended. The response phase also included funding for the Community Coalition of Youth (CCOY) to conduct a strategic planning day with the local consultant. The goal of this endeavor was to improve coordination of the CCOY, thus allowing the coalition to move beyond the discussion process and into an action-oriented effort to meet the needs of the community. It was moderately successful. CCOY was identified as a major stakeholder group and maintained involvement in COPS activities. Reported obstacles at this phase reflected a continuation of the difficulties present during the analysis phase of the project. In particular, the partnership continued to struggle with clearly acknowledging and defining the precise nature of the problem. This, in turn, lead to problems articulating specific goals and objectives. The partnership did implement a number of interventions aimed at addressing the broad issue of students in possession of/or under the influence of drugs and alcohol on/or around campus, and these responses addressed the needs highlighted by Analysis information. Yet, the partnership struggled with specifying target areas and implementing a unified, collaborative response. It is likely that, as in the analysis phase, differing missions and agendas of the participating agencies hindered this process. As a result of these difficulties, it appears that participants were left with confusion regarding roles and responsibilities. Difficulties associated with the implementation phase of community-based interventions are common (Goodman, Wandersman, Chinman, Imm, and Morrissey, 1996). Assessment The assessment phase involved designing an evaluation strategy and collecting data from partnership sources. The Project Evaluator became involved in the second year of the project, in an effort to help the partnership complete the analysis phase and select targeted project goals and objectives. Although each of the partners progressed toward responses reflective of their agency/organization’s respective analysis-related concern, a coordinated effort reflective of a unified mission statement was not achieved. As a result, responses were implemented in a staggered fashion over a period of two years. The specific evaluation plan grew out of meetings with the Project Coordinator as well as the school administrator most involved with the project. These meetings allowed for clarification of the goals of each partner aimed to achieve, as well as appropriate data available to assess progress toward these goals. The evaluation strategy includes information from the school and law enforcement sources, including: . number of school disciplinary referrals in which drug/alcohol use/sales was a central issue (expulsion and suspension); . number of citations and referrals to the Windsor Juvenile Diversion Program that focused on drug/alcohol use/sales; and . school attendance The initial evaluation strategy also involved assessing costs associated with the problem of student drug and alcohol use, using budgetary information from the school and law enforcement agency. Unfortunately, it was not possible to obtain this information due to the manner in which budgetary information is recorded. Finally, because the partnership experienced notable difficulty in moving in unity toward a response, a qualitative survey of involved partners was implemented. This survey addressed the partners’ perceptions of the role of the partnership, its visibility in the community, its accomplishments and difficulties. It was administered as an anonymous survey, in an effort to encourage feedback regarding the partnership. While surveys were distributed to 12 partners, only three were completed and returned. Thus, a truly representative sample of participant feedback was not obtained. The poor response rate for the survey appears indicative of the working relationship between the partners. In attending partnership meetings in the second year of the project, the evaluator noted a similar trend, with the same few individuals participating in the meetings. Involved parties reported that attendance was stronger during the first year of the project. It appears that perhaps initial participants, eager for a response to be implemented, withdrew during the protracted Analysis phase. While, in part, this may relate to the need to create buy-in regarding the SARA problem-solving model, it may also reflect the partnership’s ongoing difficulty mobilizing a united effort to address a clear specific mission. Initially, the program evaluator intended to use a series of chi-square tests to evaluate the impact of the response on student expulsions, suspensions and citations for substance-related offenses. The low overall rates of these behaviors resulted in a data set that limited the utility of this statistical procedure. As such, statistical significance of the analyses should be viewed with caution. In this instance, more practical significance may be gained from examining the overall descriptive trends. Impact of the Project The number of school disciplinary hearings in which drug/alcohol sales was the central issue served as a measure of the impact of the program on the school climate and offending students. This variable was further sub-divided into less severe/single offense (suspensions) and more severe/repeat offense (expulsion). Baseline data for both suspension and expulsion (1998 year) were not available. Table 1 shows the numbers of expulsions and suspensions for 1999 and 2000. Table 1. Suspensions and Expulsions for Drugs/Alcohol by Year Expulsions 1999 5 2000 1 Suspensions 1999 4 2000 5 The number of expulsions decreased from 5 in 1999 to 1 in 2000, whereas the number of suspensions remained relatively stable (1999=4; 2000=5). Although the change in expulsions did not reach statistical significance, it reflects a trend in the desired directions. These numbers suggest that the project may have been effective in its goal of decreasing repeat offending through more intensive early intervention. In keeping with this finding, there was also a reported increase in school referrals to the Youth and Family Services Bureau. To assess the impact of the project on school climate (location) and student academic-related behavior (offender/victim), average daily attendance was examined. Average daily attendance data remained stable from a baseline 1998 (93.70%) to 1999 (93.95%) and 2000 (94.44%), suggesting that the project did not influence this variable. The number of substance-related citations and referrals to the Windsor Juvenile Diversion Program was used to assess the impact of the project on both the offender and victim form a law enforcement and community perspective. Citations were analyzed in a combined fashion (total citations) as well as by isolating marijuana and alcohol citations for distinct examination. It should be noted that this is a variable particularly difficult to evaluate in a practical level, given that elements of the Response, such as the implementation of a SRO position, might serve to actually increase citations, due to increased awareness and police presence on campus. Table 2. Citations for Drug/Alcohol Use/Sales by Year Marijuana 1998 14 1999 13 2000 8 Alcohol 1998 13 1999 9 2000 9 Total 1998 27 1999 22 2000 17 Table 2 shows that the total number of citations related to drug/alcohol use/sales decreased gradually from 27 citations in 1998 to 17 citations in 2000. These differences were not statistically significant, but indicate a trend toward a reduction in overall substance-related citations. Citations specific to marijuana use/sales remained steady from 1998 to the end of the first year of the project. Following the completion of the response phase, a notable (but not statistically significant) decrease in marijuana citations occurred. Citations for alcohol use/sales declined and subsequently stabilized from 1998 to 2000. Again, this difference was not statistically significant but did show progress in the desired direction. Follow-up survey responses indicate that the project was successful in increasing community awareness of adolescent substance-use issues and encouraging a dialogue about these concerns. Given systematic issues typically related to substance-use, this is alone represents a significant accomplishment. Substance use is a complex issue in which individual risk behavior is closely intertwined with the social and cultural context in which it occurs (Goodman, Wandersman, Chinman, Imm, and Morrissey, 1996). As such, interventions that target community norms and the structure of community services are essential. The projects also reportedly increase services for at-risk youth, in terms of both on-campus and off-site counseling services. Finally, the already solid connection between the Windsor Police Department and the high school students was bolstered. To the extent that this project influenced the range, coordination and connection of substance-related community efforts, significant progress was made. Despite these gains, survey responses also indicated confusion and/or disagreement regarding what the project achieved. Respondents varied from thinking the project accomplished very little, to believing that some significant gains had been made. Feedback, both formal and informal, from participants suggests that communication, delegation and follow-through were the most significant obstacles to the project. Overall Recommendations and Conclusions As noted above, the project appears to have increased community awareness and dialogue regarding adolescent substance-use issues and increasing services for at-risk youth. Data trends suggest a decline in substance-related expulsions and citations for marijuana use/sales. The partnership appears to have had some success in the area of preventing recidivism by providing more intensive early intervention. Throughout the course of this partnership, there were notable issues related to trying to maintain and mobilize partnership relationships. While partner relationships tended to be cordial, with a united desire to help decrease the problem of “students in possession of/or under the influence of drugs/alcohol on/around campus,” the members did not appear to achieve a unified vision or sense of purpose early on in the partnering process. Communication barriers appeared related to differing agendas and ideologies, making it difficult for partners to “get on the same page.” Difficulties closely linked to the issue of substance abuse itself, such as denial, also hindered the progress. Management issues related to delegating responsibility and mobilizing follow-through meant that the partnership relied to heavily on the same individuals to get things done. This, in turn, resulted in other partners being less fully involved and, ultimately, disengaging from the process on some level. One survey respondent also suggested that the structure of the grant procedures themselves proved difficult, in that the protracted assessment phase de-emphasized an active problem-solving approach that potentially may have maintained stakeholder investment. Other respondents agreed that the process of “bogged down” targeting and analyzing substance use among students. Policy recommendations The following policy recommendations are made for future projects: 1. Attend closely to coalition readiness. Early on in the partnership, have stakeholders openly discuss agency policies, needs, and ideologies, thus facilitating a unified approach in which stakeholders’ needs are met. In addition, establish specific and clear mission and goals so that all participants are working toward common goals. Early attention to establishing organization structure and dealing with “turf issues” is essential in optimizing a coalition’s ability to build capacity for action (Goodman et al., 1996). 2. Recognize that, while partnership is highly valuable endeavor, it requires substantial effort to unite and mobilize a diverse group of participants and, as such, provide additional support, technical assistance and training for Project coordinators. 3. Utilize a third-party, as opposed to an “insider,” as Project Coordinator, particularly for projects related to a complicated issue such as substance use and abuse. While often the use of an inside source can bolster a partnership, the complex systematic issues related to substance use may be better served by a person outside the system. Recommendations to Participants Despite setbacks in the smooth functioning of the partnership, progress was made in solidifying an approach where community agencies work together to address student issues. Key players in youth development were educated regarding the SARA model and report that they will consider a non-response based, collaborative effort to address future youth service concerns. During the Analysis phase, the concept of involving students in Problem-solving efforts seemed to have caught on, and will likely be Implemented in the future. It is recommended that participants continue to build these strengths in order to continuing providing the best quality of services to all Windsor youth. Reference: Goodman, R.M., Wandersman, A., Chinman, M., Imm, P., & Morrissey, E. (1996). An ecological assessment of community-based interventions for prevention and health promotion: Approaches to measuring community coalitions. American Journal of Community Psychology, 24, 33-47.