EVALUATION OF PROGRAMS
Edging Toward Effectiveness: Examining Postsecondary Occupational Education

III. Beyond Perkins III: Other Crucial Issues

The topics suggested in Section II move beyond the provisions of the Perkins Amendments of 1998, but they do not address other fundamental issues about the context of PSOE and about the nature of federal legislation. In discussions with occupational educators and researchers, a small number of these other issues emerge again and again, and they could form the basis for other research studies.

III.1: The Meaning of Completion

By far the most vexing problem to occupational educators is the question of what constitutes completion. In conventional analyses, completion of PSOE is marked by receipt of an associate's degree-typically requiring about 60 credits including a sequence of occupational courses, related academic courses, and general education requirements-or a certificate, typically requiring about 30 credits in occupational subjects only (though some certificates are shorter or longer).17 Associate's degrees and certificates are typically defined and recognized by states, they show up on transcripts, and in my reading of course catalogues they usually require a relatively coherent group of courses. Completion has substantial economic benefits, including "program" or "sheepskin" effects (Grubb 1999); many policymakers feel it should be encouraged, and completion rates are among the performance measures required by both Perkins III and by WIA. But rates of completion in PSOE are quite low, and this has been a continual embarrassment to community colleges and other providers.

However, many students who leave PSOE without associate's degrees or certificates may nevertheless complete what they set out to accomplish. These include at least two distinct categories:

(1) Some students, particularly those needing upgrade training, intend to enroll only for a course or two. These students are more likely to be employed, to be older, and to continue in the same job. Students needing retraining, including dislocated workers, may switch occupations, but they too may enroll for only the number of courses they need to find a new position; they are also likely to be experienced workers. One way to identify these "completers" is through their intentions; some community colleges that collect data on student intentions report that large fractions of older students claim to want only a few courses. Under some circumstances, the same may be true of students needing remedial education.

(2) Students may complete credentials other than associate's degrees or certificates. These include licenses, including those required for health professions, cosmetologists and other personal service workers, some construction trades, aviation mechanics licensed by the Federal Aviation Administration; the kinds of industry-generated certificates (IGCs) created by industry associations with voluntary standards, including such groups as NATEF (the National Automotive Technicians Educational Foundation) and the American Welding Society; and some IGCs created by individual companies, such as the credentials offered by Novell and Microsoft to those individuals training specifically on their software. If students learn enough to pass such licenses and IGCs, they may leave PSOE and appear to be drop-outs when they have in reality completed a credential with, arguably, some value in the labor market. Finally, some community colleges have begun to modularize occupational programs and create shorter credentials or early-exit options, partly to recognize the reality that students leave before completing credentials, and partly to provide welfare recipients with credentials with the allowable training period. Whether these credentials have any market value is unclear, though no doubt there is local lore in specific colleges.

The fraction of "dropouts" who are really "completers" in one of these two senses is currently unknown. PSOE administrators and instructors can relate many stories about specific students who are completers in these ways, but their implication-that all PSOE students are really completers-is surely not true. Given the importance of this question to conceptions of effectiveness, as well as to information about the changing conditions of PSOE, NAVE might carry out studies specifically to examine the conception of completion.

There are at least two parts to such research. One, of course, is the investigation of student intentions. The ideal study would either create a survey of student intentions-or use one that community colleges are now using-and then verify the results through student interviews (see IV.7). The importance of interviews is that some students who are really "experimenters" may exaggerate their intentions, or may state intentions (like transfer and completion of a B.A.) that are socially acceptable but unstable;18 it's equally possible that older students searching for a substantial change in employment state intentions for completing only a few courses until they figure out the alternatives. This component of research could also try to link student intentions to information about student outcomes, either from institutional records or from UI data, to examine the match among intentions, courses and degrees completed, and employment outcomes.

The second component requires examining the receipt of licenses, IGCs, and other non-standard credentials. The problem is that the sources of information are varied. Of course, questionnaires to students about such credentials is one source of information, though response rates to student surveys are typically very low. Information about state-required licenses could come from states, and in a few cases from colleges that administer licensing exams; information about IGCs could come from the industry associations themselves. In theory these different sources of information could generate information about which individual students in a specific college and in a particular cohort have completed such credentials, and then UI data or other follow-up data could be used to ascertain their subsequent employment and earnings. One alternative is to choose a few community colleges in relatively well-defined labor markets, in states with adequate UI data, and use them as test cases to develop both the methodology of track alternative credentials and their effects on employment.19

The development of IGCs presents still another issue for NAVE to address. In this country, interest in skill standards has led to a series of pilot projects funded by the Department of Labor to create standards in 22 industries. However, IGCs are evidence of industry-generated standards (or IGSs) that have already been created by industry associations who feel a need for such standards and credentials, rather than being artificially created by government sponsorship. A study of such IGSs and IGCs could focus on a limited number of such industry groups, ascertain how widespread such standards and credentials are, how widely they have been adopted by community colleges and other PSOE providers, what their use by employers is, and what value they have in hiring, promotion, wage setting, and other aspects of employment.20 The appearance and the apparent expansion of IGSs and IGCs provides a natural experiment in skill standards, one that could shed considerable light not only on the developing conditions of PSOE but also on the conditions under which skill standards are likely to be effective.

III.2: The Environment of PSOE: The "New Fluidity"

There's a new day dawning in PSOE, many observers report. Students accumulate credits from many different institutions, including four-year and two-year colleges; they may learn skills from company-sponsored courses or IGCs, or though the military or experience, or on the Internet or through distance learning so that the spatial limitations of conventional providers are irrelevant. Employers care about skills, not credentials, so the accumulation of skills from different sources, without credentials attached to them, is still effective in providing access to employment. In this fluid environment, the notion of entering a "two-year" college, attending full time, progressing steadily through the requirements for general education and a major, and receiving an associate's degree in roughly two years is a rare event, an image of the past (Schneider and Stevenson 1999, Ch. 9). In the "new fluidity"-to give it a name-students draw on many providers of education and training, and the educational programs so carefully put into place by community college instructors debating the coherence of their offerings are all but irrelevant. The "new fluidity"-which may not be so new, after all-is sometimes invoked to explain why there are so few completers in community colleges, and sometimes used to describe a world of fierce competition among providers, where community colleges may lose their "customers" to the Internet, distance learning, other public and private providers.

However, it isn't clear how realistic this portrait is, or how extensive the mix-and-match practices of the "new fluidity" are.21 It isn't clear how students signal their skills to employers, for example, or how the pattern of earning credits from many providers can teach the competencies like SCANS skills and other higher order competencies that some employers say they want. Sometimes this scenario seems like alarmism, particularly of educators fearful of losing students, or concerned that their carefully planned programs are being trashed by student practices, and the reality might not match their fears. One option, therefore, is to explore the "new fluidity" empirically, particularly through community case studies (see IV.5 below) which would examine the variety of education and training providers in several communities — including distance and Internet providers — to see the extent to which students move among them. In addition, it would be critical to interview a sample of students (see IV.7), to see how they are putting together their own programs. This would, of course, be complementary to interviews with students intended to determine their goals in enrolling in community colleges, and whether from the outset their intentions include gathering credits from numerous sources.

If the "new fluidity" proves to be an accurate description of the direction of PSOE, it raises questions about the appropriate role of publicly funded institutions. It's possible that some kinds of education are best left to private providers or to distance learning and the Internet — for example, information-dense subjects for independent students. PSOE may have a different niche — for example, subjects where the integration of workshop and classroom learning is important, or where constructivist or systems approaches to teaching higher order competencies are important. But before it's possible to say what the role of PSOE could be in the "new fluidity," it's important to understand better what precisely is taking place.

III.3: The Limited Success of Federal Legislation

Federal legislation related to vocational education, school-to-work programs, job training, adult education, and other aspects of work force development has had, in my view, only limited success. On the one hand, it has provided support for some good ideas — for example, the upgrading of vocational education, the integration of vocational and academic learning, the incorporation of work-based or service learning with school-based learning, the efforts to link secondary and postsecondary education, the development of yet other second-chance programs for the long-term unemployed and welfare recipients. Virtually every innovation in PSOE is funded in some way or other with federal funds, and the impact of federal funding often seems much larger than the amount — "the tail wags the dog," in the conventional expression. On the other hand, the spread of these practices has been slow, partial, and incomplete. A great deal of federal funding for occupational education is spent for routine forms of program improvement — for example, upgrading equipment and courses, which is something that occupational programs ought to be doing routinely — and for relatively standard forms of remediation. Perkins II fostered a great deal of curriculum integration, but largely in the form of applied academics courses — the most pallid form of integrating academic and vocational education. Tech prep and school-to-work funds have been used to support bits and pieces of good practice-some articulation agreements, some additional guidance and counseling, some applied academics courses — but without doing much to create coherent systems with widespread participation (Hershey, Silverberg, Owens, and Hulsey 1998; Hershey et al. 1998).

Before examining the final question of what the federal government should do in PSOE, in the next section, it's important to raise the issue of why federal legislation in this arena has been so checkered in its effects — or, more precisely, to see whether there's some empirical research that might shed some light on this issue. There are several hypotheses and fruitful avenues for research:

  • Many occupational educators report that state and local funding is simply inadequate for their programs, since there is (in most states) no recognition of the higher costs of certain occupational courses, of the special equipment and materials costs, and of the ancillary costs of establishing connections with employers including work-based placements. Therefore they have to scrounge for every additional dollar they can find, and federal funds are used in these ways too. The implication is that in states that do have funding for the higher costs of occupational programs, and routinized funding for capital equipment, it might be possible that federal funds are used in different ways.

  • The amount of federal funding in most PSOE institutions is very small — a dated estimate is that perhaps one to three percent of overall funding in community colleges, or two to four percent of funding for occupational education come from federal sources (Grubb and Stern 1989). With such small amounts of funding, programs cannot commit themselves to larger changes that require state and local funding — and so federal funds are used for "bits and pieces" of programs, not coherent system changes. One implication is that programs that receive relatively larger amounts of federal funding may use them in distinctly different ways.

  • Outside funds are often viewed differently than internal funds, since they may disappear, or have separate regulations attached to them that require administrators to keep them separate, or have participation and allocation requirements different from other funds. This usually means that outside funds are reserved for relatively peripheral activities rather than being integrated into mainstream programs.

These hypotheses, and others that could be generated through discussions with federal and local administrators, could be investigated by examining spending patterns and through interviews with local administrators. Some of these questions could be integrated into questionnaires or interviews with state directors of PSOE (see IV.2) and into questionnaire studies and case studies (IV.3 and IV.4) of local providers. Alternatively, it might be possible to identify some PSOE institutions that have used federal funds in distinctly different ways, and then identify what conditions of these institutions have influenced this difference.

This question is, of course, much larger than federal legislation for occupational education and related programs. Many federal programs-and indeed most state programs — intended to enhance the effectiveness of social programs including education have been faulted for low effectiveness in instigating reform. However, before deciding what future directions for policy might be, it is crucial to establish when federal legislation has been effective and when its influence has been only marginal.

III.4: What Should Federal PSOE Policy Be?

The culminating question, the one that in many ways motivates all the research that NAVE carries out, is what PSOE policy might be, and whether it should take a new direction in legislation subsequent to Perkins III. The question here is not what this direction might be, but whether there are research activities that might help determine the options for federal policy.

There are at least five activities that might be helpful in addressing this question:

  • A review of the principles of fiscal federalism, which specifies when a federal government might provide goods and services and when a state or local government is more appropriate.22 Some understanding of the strengths and weakness of state policy in PSOE, from IV.3 above, might be useful in this exercise.

  • A review of the principles and results of federal legislation in other areas of education and job training, including the effectiveness of such policies. For example, policy in K-12 education has tended to provide funding through categorical programs like Title I rather than through student funding as in postsecondary policy.

  • An analysis of other countries and their efforts to create PSOE policies might be useful, particularly if the examination concentrated on countries reasonably like our own-for example, Australia with its federalist system, Great Britain with its free-market approaches-rather than countries like Germany and Japan with very different institutional and government histories.

  • A consensus-generating exercise might be useful, bringing together a variety of federal, state, and local administrators and policymakers, to identify the current strengths and weaknesses of different levels of government. This exercise might start, for example, with a paper outlining a broad array of options for federal policy — or for a joint conception of federal and state policy-based on other activities, and then have participants weigh the alternatives from a variety of perspectives.

  • All NAVE contractors could be required to include a section on implications for federal policy at the end of their reports. This will at least force individuals to question what potential implications there might be of the very different kinds of information they collect.

In general, these activities seem pallid in comparison with the task of specifying new directions for federal policy. This is perhaps because the broad outlines of policy are not usually created from rational exercises like this, but rather from a more extended process of discussion, debate, and consensus.23 However, because there has been so little explicit thought given to what federal policy in the arena of PSOE should be, NAVE could at least begin the discussion through the activities I have outlined.


[ Go to ED Home Page ]

This page last modified August 11, 1999 (glc).

 
Print this page Printable view Send this page Share this page
Last Modified: 11/04/2003