A r c h i v e d  I n f o r m a t i o n

National Evaluation of The Even Start Family Literacy Program, 1998

Chapter 7

What Were the Parenting Outcomes?

Outcomes for parenting education were assessed by the HOME Screening Questionnaire (HSQ) (Coons et al., 1981), which replaced the set of questions about parent-child activities and the home environment contained in the first Even Start evaluation. The selection of new measures for assessing parent-child interactions reflected concern about participant and project-level burden, as well as concern that measures used in the first evaluation did not adequately assess the behaviors of most interest to Even Start. The parent-child interview used in the first study suffered from high pretest means on some sub-scales and small gains on most sub-scales. Although the interview from the first evaluation was based on three key measures including the HOME, staff identified questions that required rewording and asked for guidelines for excluding families based on the age of the child. The quality of local administration in that evaluation was highly variable. Given these problems and based on recommendations from the first Even Start evaluation's Advisory Panel, we decided to assess parenting skills by replacing the questions used in the first evaluation with the HSQ.

Home Screening Questionnaire

The HSQ is a survey version of the Home Observation for Measurement of the Environment (HOME; Caldwell and Bradley, 1984), which can be administered either in the home or in a center. The HSQ covers many of the same topics as the HOME but gathers data through parent self-report rather than direct observation.

The HOME is an accepted measure of the quality of cognitive stimulation and emotional support provided to the child by the family. Based on observation of the home environment during a visit to each family's residence, it includes some open-ended interview items and requires more than an hour to complete. The HOME has been widely used in large-scale research studies; scores on the HOME are related to concurrent child performance on standardized cognitive measures and to later academic performance. Psychometric analyses indicate that the HOME has adequate reliability.

Researchers at the University of Colorado Medical School developed the HSQ in an effort to offer a briefer instrument that taps similar constructs with simpler data collection requirements. The HSQ is a parent-answered questionnaire written at a 3rd- or 4th-grade level. It consists of multiple choice, fill-in-the-blank, and yes/no questions and takes fifteen to twenty minutes to complete. The HSQ scoring was standardized on a sample of 1,500 low-income families. Psychometric studies have shown that the HSQ is highly correlated with the HOME total score.¹⁰²

Two forms of the HSQ are available, based on the age of the child: birth up to 3 years and 3 to 6 years. The instrument for younger children has thirty items and a toy list, while the instrument for older children has thirty-four items and a toy list. An administration manual provides rules for scoring each item to yield a total score. Total scores range from zero to forty-three on the form for younger children and zero to fifty-six on the form for older children. Both forms were translated into Spanish by Abt staff.

In the Sample Study, the HSQ was administered to one parent per family by Even Start staff. The questions were asked in reference to one child in the family; if two or more children were expected to participate in Even Start, staff were asked to select the child expected to have the greatest involvement in the program.¹⁰³ Project staff were trained on the HSQ during the summer of 1994. A total score was calculated and used in the analyses reported here; information on individual item scores was not obtained.

Performance on HSQ

For the parents of children less than 3 years of age in the Sample Study, the average pretest on the 0-3 version of the HSQ was 30.1, with a standard deviation of 7.2 (see Appendix D, Exhibit D.3). Pretest means for all parents (whether or not those parents were also posttested) increased for those parents who had completed additional years of education, as might be expected. The average scores also were somewhat higher for parents whose primary language is English (30.0) than for parents whose first language is not English (27.7). A larger group completed the HSQ about a child between 3 and 6 years of age. Among this group, the average pretest score was 35.1, with a standard deviation of 7.6 (see Exhibit D.4 in Appendix D). As is the case with parents of younger children, parents with more education and parents whose primary language is English tended to score higher on the measure.

The average amount of time between pretest and posttest was approximately seven months. Among parents whose children were less than 3 years old, there was a gain of 3.6 points from pretest to posttest, corresponding to a standardized gain of .62 (Exhibit 7.6). A nearly identical gain (3.5 points) was observed for parents of older children, which corresponds to a standardized gain of .50. Both gains are considered moderate for program evaluations in the social sciences.

We explored associations and relationships between the HSQ and key predictors, using simple correlations first and subsequently building simple and multiple regression models. We examined the relationship between the HSQ and such predictors as the amount of parenting education hours, amount of parent and child joint time, family need index, and project staff characteristics (e.g., number of families served during program year, number of staff, percent of staff with BA or higher degree, staff-to-family ratio, and staff experience). We found that only pretest score remained a significant predictor of gain score, for both versions of the HSQ. Pretest scores account for 30 percent of the variation in gain scores for the parents of children between 0 and 3 years of age, and for 20 percent of the gain score for parents of children between 3 and 6 years of age. The regression coefficients for these (and all adult outcome measures) are displayed in Exhibit 7.7, and a table displaying correlation coefficients is displayed in Appendix D, Exhibit D.9.

Gains Compared to Other Studies

The gains on the HSQ are encouraging, and are larger standardized gains than were generally seen for the parent-child items in the earlier Even Start evaluation. Although there is no control group or norms group for the HSQ, one way to assess the size of these gains is by comparing them to the gains observed for the control group in a separate evaluation of a very large demonstration program for low-income families. The Comprehensive Child Development Program (CCDP) is a family-support, two-generation program supported with federal funds from the Administration on Children, Youth and Families within the U.S. Department of Health and Human Services. As part of the national CCDP evaluation, the HSQ was administered to a sample of low-income control group parents when their children were 18 and 30 months of age. In the control group, the HSQ scores were virtually identical for children when they were 18 months as they were for the same children at 30 months of age (the mean scores were 20.8 and 20.2, respectively).¹⁰⁴ These scores suggest that we might not expect any "normal or developmental" growth over time in the HSQ scores for low-income families. They further suggest that the changes observed in the HSQ scores for Even Start families may be attributable to participation in Even Start, rather than to other factors.

Footnotes:

¹⁰² Test-retest correlations for the HSQ over the two weeks time range from .62 to .86 with internal consistency coefficients between .74 and .80 (Frankenburg and Coons, 1986).

¹⁰³ This method of selecting children for the HSQ represents a limitation in the data collection forms because the forms allow reporting only for one child. Consequently, the HSQ scores may be biased.

¹⁰⁴ HSQ scores for CCDP control group children were obtained from age-specific assessments (i.e., when the children were 18 months and 30 months of age), while HSQ scores for Even Start children were obtained at specific points during the program year, regardless of the child's age. As a result, the scores are not directly comparable across the two studies.

-###-

[ What Were the Child Development Outcomes? ]

[ What Were the Adult Education Outcomes? ]