Effectiveness

Findings

The WWC beginning reading review addresses student outcomes in four domains: alphabetics, fluency, comprehension, and general reading achievement. ⁸ Studies included in this report cover three domains: alphabetics, comprehension, and general reading achievement. Alphabetics includes five constructs: phonemic awareness, phonological awareness, print awareness, letter knowledge, and phonics. Comprehension includes two constructs: reading comprehension and vocabulary development. General reading achievement includes outcome measures that do not explicitly differentiate among different reading domains (e.g., a summary standardized test score). The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects on students. ⁹ The results are presented by domain for each of the SFA® studies that met the WWC evidence standards with or without reservations.

Alphabetics
In the alphabetics domain, seven studies addressed phonics outcomes and one of these studies also measured students’ letter knowledge skills.

Three years of program implementation:

• Borman et al. (2006) examined scores on the Woodcock Reading Mastery Test (WRMT) and reported statistically significant positive effects for two phonics subtests: Word Identification and Word Attack. The WWC analysis confirmed the statistical significance of these effects.
• For each SFA® school,¹⁰ Madden et al. (1993) found statistically significant positive effects on the phonics measure (the Woodcock Language Proficiency Battery (WLPB) Word Attack subtest) for preschoolers and first-graders and statistically significant positive effects on the WLPB Letter-Word Identification subtest for kindergarteners. The WWC found that none of the combined effects across schools were statistically significant, but the average effect size across these outcomes was substantively important according to WWC criteria (that is, an effect size of at least 0.25).

Two years of program implementation:

• Dianda and Flaherty (1995) reported effect sizes, but did not report on the statistical significance of the effect of SFA® on two phonics measures: the WLPB Letter-Word Identification subtest and the Word Attack subtest. According to WWC calculations, there were no statistically significant effects of SFA®, but the average effect size across the two measures was positive and large enough to be considered substantively important.
• Ross and Casey (1998) reported no statistically significant effect of SFA® for one phonics measure (WRMT Word Identification subtest) but found a statistically significant positive effect for the other phonics measure (WRMT Word Attack subtest). In WWC computations, neither of the effects was statistically significant, and the average effect was not large enough to be considered substantively important.

One year of program implementation:

• Ross, Alberg, and McNelis (1997) did not find a statistically significant effect of SFA® for one phonics measure (the WRMT Word Identification subtest), but did find a statistically significant positive effect for the other phonics measure (WRMT Word Attack subtest). The WWC analyses showed that neither of the effects was statistically significant. In addition, the average effect size across the two outcomes was neither statistically significant nor large enough to be considered substantively important.
• Ross et al. (1998) study found no statistically significant effects of SFA® on the two phonics outcomes: WRMT Word Identification and Word Attack subtests. The WWC analyses also found that no effects were statistically significant, but the average effect size across outcomes was positive and large enough to be considered substantively important.
• Smith et al. (1993) reported no statistically significant effect of SFA® on the letter knowledge construct (WRMT Letter Identification subtest), but found statistically significant positive effects for the two phonics outcomes (WRMT Word Identification and Word Attack subtests) for first-grade students. For kindergarten students, the authors found statistically significant positive effects for the WRMT Letter Identification and the Word Identification subtests. The WWC calculations found that although none of these effects were statistically significant, the average effect size across outcomes was positive and large enough to be substantively important.

Overall, in the alphabetics domain, one study with a strong design showed statistically significant positive effects. Four studies showed substantively important positive effects and two studies showed indeterminate effects. ¹¹

Comprehension
In the comprehension domain, six studies addressed reading comprehension outcomes, and one of these studies also measured students’ vocabulary development skills.

Three years of program implementation:

• Borman et al. (2006) reported and the WWC confirmed a statistically significant positive effect of SFA® on the WRMT Passage Comprehension subtest.

Two years of program implementation:

• Dianda and Flaherty (1995) did not report on the statistical significance of the effect of SFA® on the WLPB Passage Comprehension subtest. The WWC found no statistically significant effect, but the positive effect was large enough to be considered substantively important according to WWC criteria.
• Ross and Casey (1998) reported no statistically significant effect of SFA® on the WRMT Passage Comprehension subtest. In addition, the WWC found that the effect size was positive, but not substantively important.

One year of program implementation:

• Ross, Alberg, and McNelis (1997) reported no statistically significant effect on the WRMT Passage Comprehension subtest and the WWC found that the effect size was positive, but not substantively important.
• Ross et al. (1998) reported and the WWC confirmed a positive, but neither statistically significant nor substantively important effect of SFA® on the WRMT Passage Comprehension subtest.
• Smith et al. (1993) reported no statistically significant effect of SFA® on the vocabulary development measure (Peabody Picture Vocabulary Test) for kindergarteners. For first-graders, the study authors found a statistically significant positive effect on the WRMT Passage Comprehension subtest. The WWC analysis found that none of the effects were statistically significant; and the average effect size across all outcomes was not large enough to be considered substantively important.

For the comprehension domain, one study reported a statistically significant positive effect and had a strong design. One study showed substantively important positive effects, and four studies showed indeterminate effects.

General reading achievement
Six studies examined outcomes in the general reading achievement domain.

Three years of program implementation:

• Dianda and Flaherty (1995) examined the effects of SFA® on the combined measure of WLPB and Durrell Oral Reading subtest for three cohorts of students after two to four years of program implementation. The authors did not report on the statistical significance of the findings. The WWC effect size computations found that although none of the effects was statistically significant, the mean effect size across all outcomes was positive and large enough to be considered substantively important.
• For each SFA® school, 10 Madden et al. (1993) found statistically significant positive effects of SFA® on the Durrell Oral Reading subtest for kindergarten and first-grade students. The WWC computations found that none of the positive effects combined across schools were statistically significant; but the mean effect across grade levels was large enough to be considered substantively important.

Two years of program implementation:

• Ross and Casey (1998) reported and the WWC confirmed a positive but neither statistically significant nor substantively important effect of SFA® on the Durrell Oral Reading subtest.

One year of program implementation:

• Ross, Alberg, and McNelis (1997) reported and the WWC confirmed a positive but neither statistically significant nor substantively important effect of SFA® on the Durrell Oral Reading subtest.
• Smith et al. (1993) found a statistically significant positive effect of SFA® on the Durrell Oral Reading subtest. The WWC computations found that the effect was not statistically significant, but large enough to be considered substantively important.
• The Ross et al. (1998) reported and the WWC confirmed a positive, but neither statistically significant nor substantively important effect on the Durrell Oral Reading subtest.

In the general reading domain, three studies reported substantively important positive effects and three studies showed indeterminate effects. No study had a strong design.

Rating of effectiveness

The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies (see the WWC Intervention Rating Scheme).

Top