Assessing the Teaching and Student Learning Outcomes of the Katz/Henry Faculty Development Model

Purpose

The Katz/Henry model of faculty development, Partners in Learning (PIL), evolved during a decade-long collaboration between Joseph Katz and Mildred Henry which was supported by one of FIPSE's first grants in the early 1970s. In the late 1980s PIL, financed by the New Jersey Department of Higher Education and administered by the Woodrow Wilson National Fellowship Foundation, was disseminated among public and private postsecondary institutions in New Jersey. Now PIL is housed in the New Jersey Institute for Collegiate Teaching and Learning, at Seton Hall University. Over the last decade, PIL programs have operated on 27 campuses in New Jersey and involved over 500 faculty.

Fully described in Katz and Henry's 1998 book, Turning Professors into Teachers, PIL is unique among faculty development methods in that it focuses on student learning and is based on classroom interactions. Concerned with process rather than goals, the model is faculty-run, non-judgmental, and encourages innovation and exploration. It seeks to reinvigorate faculty and alleviate the stresses and isolation of the classroom.

Faculty pair off and exchange observer-observed roles in the classroom on a term-by-term basis. The faculty pairs meet frequently, interview students several times, and join other participating colleagues for general discussions of teaching-related matters.

According to prior evaluations of Katz/Henry programs, faculty felt that they had made significant improvements in their teaching, but these self-reports could not be verified. The increasing concern with the quality of instruction and faculty development, combined with decreasing financial resources at most institutions, prompted the director of this project to try to find out whether there was in fact a link between the PIL approach to faculty development and student learning.

Innovative Features

Twenty-four faculty from four institutions—Rutgers University, Newark, Bergen Community College, the County College of Morris, and Kean College—participated in the assessment. Mostly Ph.D. holders, they came from all ranks and averaged 19 years of teaching experience, ranging from six to forty years. Some had considerable faculty development experience, while others had only recently become interested in improving their teaching. Still others appeared somewhat disinterested but were motivated by the stipend and release time offered to participants.

Faculty were coupled in three interdisciplinary pairs per institution, typically a social scientist and a natural scientist. Each pair's work revolved around a focal course—an introductory-level offering that was part of the observed instructor's regular load. The partners interviewed a minimum of three students three or four times per semester, kept a journal of their observations, and attended monthly sessions at the New Jersey Institute for Collegiate Teaching and Learning to discuss their experiences and explore innovative pedagogies.

Two of the three pairs on each campus were active in years one and two and fallow in three. The remaining pair (the control) was fallow in year one and active in years two and three. Data were collected on fallow pairs in year one to establish the baseline control and on fallow pairs in year three to determine the persistence of change.

Students were surveyed with pre- and posttests regarding their preference for class structure and teaching style, their interactions with faculty and their experiences in the course, faculty teaching behaviors and their impact on learning, personal and intellectual changes resulting from the course, and progress in academic skills. The posttest also included an academic achievement question—graded anonymously by the course instructor—to reflect the students' mastery of the subject matter. Responses to this question were controlled for level with the students' GPA and college placement test scores.

Project staff observed faculty in the classroom a minimum of three times per semester. They looked for clarity, enthusiasm, organization, and pacing of the delivery, and interaction and rapport with students. Course syllabi were collected and analyzed, and each participant was interviewed. In addition, faculty completed two attitudinal surveys closely resembling the students' posttest.

Project Impact

Most faculty believed that as a result of PIL they had increased the diversity of their teaching strategies-especially using more active learning approaches—and their awareness of their influence over students. They also stated that because of the program they understood better how students learn and improved their relationships with colleagues as well as their teaching.

Students agreed that their instructors had changed, citing more and better interactions with them and higher academic expectations. But they reported perceiving no significant changes in in-class teaching behaviors or personal or intellectual changes as a result of being in the course. There were no discernible patterns of increased academic achievement in the students of participating faculty, even when controlling for ability.

Using informal interactions with faculty, the journals, and the exit interviews, staff plotted each faculty participant on a grid illustrating the perceived degree of change in teaching and the amount of personal reflection about the process. Three clusters emerged—those at the low end of the scale, labeled "resisters," those in the middle of the grid, the largest number, labeled "moderate changers," and those at the upper end of the scales, labeled "converts." Staff then used the students' and faculty's self-reports to test these designations. On the whole, the typology was not supported: distinguishing between faculty types based on the students' responses or the faculty's self reports was, with few exceptions, impossible.

Lessons Learned

Project staff speculate that the inconclusive results of the assessment may be due to the failure to select the appropriate outcomes. Faculty enter a project such as this one with highly individual and idiosyncratic goals, and these goals are not likely to be reflected in the few general outcomes selected for assessment. Another important factor is the nature of the population—the faculty who chose to undertake PIL were for the most part committed to good teaching and thus less likely to show dramatic growth than a randomly-selected group.

A number of factors made implementation difficult. To maintain PIL's non-evaluative tone, it is important that faculty set their own agenda and rules, with minimal intervention from the project coordinator. In an assessment situation, however, this makes control of the "treatment" all but impossible. During the project, events such as sabbaticals, illness, changes in teaching load, and schedule and personality conflicts conspired against consistency.

It was challenging to keep faculty motivated to participate and furnish data over three years. But it was even more challenging to collect academic data from students, who were not especially interested in providing it and received no encouragement from their instructors. Faculty were reluctant to take up course time with project-related activities, and, despite the fact that students signed a consent form, faculty members said they felt that it was unethical to ask them to provide academic data that were not relevant to the course.

The project generated massive amounts of data from different sources and in different forms. These were hard to analyze in a meaningful way, especially when quantitative and qualitative data differed on the same issue.

Available Information

Further information may be obtained from:

Martin Finkelstein
College of Education and Human Services
Seton Hall University
South Orange, NJ 07079
Telephone: 973-275-2056

[Cold Spring Harbor Lab.] [Table of Contents] [Southwest Texas State Univ.]

FIPSE Home