A r c h i v e d  I n f o r m a t i o n

A First Look at What We Can Learn From High Performing School Districts: An Analysis of TIMSS Data From the First in the World Consortium, August, 1999

How Well Did the FiW Consortium Perform When Benchmarked Against an International Measure of Math and Science Achievement?

As mentioned earlier, one of the FiW Consortium's three primary goals was to benchmark its achievement against an international measure of student achievement. The FiW chose TIMSS as its measure. FiW student assessments were administered during 1996, and preliminary results were made available in January 1997. This section discusses the results of these assessments.

In general, FiW students did exceedingly well on TIMSS, particularly in the fourth and eighth grades. In the twelfth-grade advanced math and physics tests, the FiW Advanced Placement (AP) students also scored among the top performing countries. FiW students performed among the highest performing nations on the twelfth grade general knowledge achievement tests and near the international average on the advanced math and physics tests.

Fourth-, eighth-, and twelfth-grade results are discussed below and reported in more detail in appendix B.

Fourth-Grade Results

Only students in Singapore had scores significantly above those of FiW students on the fourth grade math assessment. The FiW Consortium had average scores that were not significantly different from four other countries (Korea, Japan, Hong Kong, and Netherlands). FiW students outperformed their counterparts in 21 of 26 countries.

In science, no nations outperformed fourth- grade students in the FiW. The FiW fourth grade science score was not significantly different from that of one other country (Korea). FiW students outperformed their counterparts in 25 of 26 countries. See exhibits B-1 and B-2 for more detail on scores and distributions.

Although FiW students did well on the fourth grade math test, they had stronger performance in some topic areas than in others. FiW fourth-grade math students performed among the best in the world in 8 of 14 content areas.11 They were:

Topics where students from other nations scored higher were:

For more detail on how the FiW performed relative to other TIMSS countries, see exhibit B-3.

Eighth-Grade Results

Eighth-grade FiW students also performed very well on the TIMSS assessment compared to students from the 41 countries that participated in this part of the study. As in the fourth grade, only students in Singapore outperformed eighth-grade FiW students in math and no nations outperformed FiW students in science.

In math, only students in Singapore scored significantly above FiW students on the eighth-grade assessment. FiW had an average score that was not significantly different from six other countries (Korea, Japan, Hong Kong, Belgium-Flemish, Czech Republic, and Slovak Republic). FiW eighth-grade math students outperformed their counterparts in the remaining 34 of 41 countries.

In science, no nation outperformed FiW on the eighth-grade assessment. The FiW score was not significantly different from eight other high-performing countries (Singapore, Czech Republic, Japan, Korea, Bulgaria, Netherlands, Slovenia, and Austria). See exhibits B-4 and B-5 for more detail.

As in the fourth grade, eighth-grade math achievement varied by topic. FiW eighth-grade math students performed among the best in the world in 9 of 20 content areas.12 They were:

Students from other nations scored higher on the following topics:

See exhibit B-6 for more detail.

Twelfth-Grade Results

In the twelfth grade, two sets of assessments were administered to the countries participating in TIMSS. The first set measured student achievement in general math and science knowledge. The second was designed to measure achievement of the most advanced students in their final year of secondary school. Accordingly, the advanced math exam covered advanced math topics, including geometry, numbers and equations, and calculus. The advanced exam in science focused on physics.

In the United States, the advanced math assessment was given to students who had taken, or were taking, a full year of a high school course that included calculus in the title, including calculus, pre-calculus, AP Calculus, and calculus and analytic geometry.

Using the U.S. definition for advanced math, approximately 14 percent of the U.S. school-leaving age cohort was covered by the TIMSS sample of advanced math students. Internationally, 19 percent of the school-leaving age cohort was covered, under the various definitions used by different countries to identify their most advanced students.

By comparison, the FiW advanced math sample covered approximately 65 percent of the FiW school-leaving age cohort under the same definition used by the U.S., clearly a much larger percentage of students than the U.S. or its international peers.13

To take the physics assessment, U.S. students had to be enrolled in, or have taken at least one year-long class of physics (this includes physics and AP Physics). Under this definition, approximately 15 percent of the U.S. school-leaving age cohort was covered by the TIMSS sample of physics students. Internationally, approximately 13 percent of the school-leaving age cohort were covered by this assessment using the different definitions of eligibility developed across countries.

As with mathematics, a much larger percentage of FiW twelfth-grade students were exposed to physics than U.S. or international students. In the FiW, approximately 67 percent of the school-leaving age cohort were covered by the physics sample. 14

The large difference between the percent of the school-leaving age cohort covered in the U.S. and FiW samples is explained primarily by differences in course taking patterns, rather than differences in the number of students in this age cohort who are still in school or differences in which components of the system they may have excluded from their sample.

Eighty-three percent of FiW students take mathematics and 74 percent take science in their last year of schooling. In the U.S., however, less than two-thirds of all seniors enroll in a math class and less than one-half of U.S. seniors take a science class.

Furthermore, nearly all FiW students take at least one of the following classes: pre-calculus, calculus, AP Calculus, physics, or AP Physics.15 Results for each set of exams are discussed below.

General Knowledge

In general mathematics knowledge, FiW twelfth grade students' performance was not significantly different from students in the seven highest performing countries (Netherlands, Sweden, Denmark, Switzerland, Iceland, Norway, and Australia).

FiW students outperformed their counterparts in the remaining 14 countries. It should be noted that no Asian countries participated in the end of secondary school assessments. See exhibit B-7 for more detail on the countries participating in this assessment and the distribution of their scores.

In general science knowledge, the achievement of FiW twelfth-graders was similar to their achievement in mathematics. FiW twelfth-grade students' performance on general science knowledge was not significantly different from students in the seven highest performing countries (Sweden, Netherlands, Iceland, Norway, Canada, New Zealand, and Australia). FiW students outperformed their counterparts in the remaining 14 countries. See exhibit B-8 for more detail.

Advanced Math and Physics

The TIMSS advanced math and physics exams were designed to be administered to the highest performing students in the world in math and science as they are about to leave secondary school. Each country defined the groups of students that they thought most appropriate to be included in these assessments, based on the general content of the tests as well as practical considerations.

In order to have large enough samples of students taking the advanced math and science exams, the U.S. included pre-calculus and general physics students in the advanced groups. Other countries limited their testing to calculus and advanced physics students.

The Consortium followed the U.S. sampling parameters, even though it had a large proportion of students who would qualify under the more restrictive international criteria, as discussed below. We have, therefore, presented two different views of the advanced math and physics test data.

In advanced mathematics, for example, we present the FiW Advanced Placement Calculus student score on the advanced examination to provide comparison with other countries. We also provide the score for all students who took the advanced test for comparison to the U.S. national score.

As discussed in the rest of the section, the Consortium's AP Calculus and AP Physics students were first in the world. However, when the pre-calculus and general physics students are incorporated into the FiW scores, the FiW averages drop below the international average. The section considers the general results first, then provides the results for the AP Calculus and AP Physics students.

On the advanced math assessment, FiW students scored near the international average. Advanced FiW twelfth-grade math students were outperformed by students in seven countries (France, Russian Federation, Switzerland, Denmark, Cyprus, Lithuania, and Sweden). FiW scores were not significantly different from those of six countries. FiW placed significantly above three countries. See exhibit B-9 for more detail.

FiW physics students were significantly below students in twelve nations (Norway, Sweden, Russian Federation, Denmark, Slovenia, Germany, Australia, Cyprus, Switzerland, Greece, Canada, and France). FiW performance did not differ significantly from three nations, outperforming only the United States in the TIMSS physics assessment. See exhibit B-10 for more detail.

AP Calculus and AP Physics Results

Many have wondered whether it was appropriate to be comparing two-thirds of the students in FiW against one-fifth or fewer students in the U.S. or internationally because such large percentages of FiW twelfth grade students were included in the advanced math and science samples. It has been suggested that AP Calculus or AP Physics students might make a better comparison group for the advanced math and science assessments because these students are enrolled in the most advanced courses and similar percentages of FiW and international students would be covered under such a comparison.

AP courses are offered in all FiW high schools and are generally considered to be the most advanced classes offered in mathematics or physics, since students participate in national AP exams and may receive college credit if they score well on these exams. FiW AP Calculus students represent 28 percent of the FiW school-leaving population. Under this comparison, then, 28 percent of the FiW students are being compared against 19 percent of international students and 14 percent of their U.S. counterparts.

In AP Physics, 7 percent of the FiW school-leaving cohort is compared against 13 percent internationally and 15 percent of U.S. students.16

FiW twelfth-grade AP Calculus students were first in the world, as no nations outperformed FiW AP Calculus students on the advanced math assessment. The FiW score was significantly above that of sixteen nations. See exhibit B-11 for more detail.

Performance of AP Physics students in FiW Consortium was also first in the world. No nations scored significantly above FiW AP Physics students. Five nations (Norway, Sweden, Russian Federation, Slovenia, and Germany) had scores that were not significantly different from FiW AP Physics students. FiW AP Physics students outperformed their counterparts in 11 countries. See exhibit B-12 for more detail.

Summary

When benchmarked against an international measure of math and science achievement, FiW students performed exceptionally well in all grades tested.

FiW students excelled on the fourth, eighth, and twelfth grade general knowledge tests, and scored among, or just below, the highest performing countries worldwide. Although all Consortium students tested on the advanced math and physics tests did not perform as well as expected (they scored around the international average), the performance of AP students was exceedingly high, with their scores placing them in first place internationally.

The outstanding performance of the FiW students, particularly given the disappointing results of their U.S. counterparts, has generated a lot of interest in examining what factors might have contributed to this world-class performance.

The next two sections look at the influence that home factors and differences in the contexts for teaching and learning might have on these gaps.

-###-



[ What is the FiW Consortium? ] 		[ Table of Contents ]
[ Do Home Factors Explain the High Achievement of the FiW Consortium? ]

This page last modified on November 23, 1999. (dtm)