[Federal Register: June 3, 1996 (Volume 61, Number 107)]
[Proposed Rules]
[Page 27989-27993]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[[Page 27989]]
_______________________________________________________________________
Part V
Department of Education
_______________________________________________________________________
34 CFR Part 701
Standards for Conduct and Evaluation of Activities Carried Out by the
Office of Educational Research and Improvement (OERI)--Designation of
Exemplary and Promising Programs; Proposed Rule
[[Page 27990]]
DEPARTMENT OF EDUCATION
34 CFR Part 701
RIN 1850-AA52
Standards for Conduct and Evaluation of Activities Carried Out by
the Office of Educational Research and Improvement (OERI)--Designation
of Exemplary and Promising Programs
AGENCY: Department of Education.
ACTION: Notice of proposed rulemaking (NPRM).
-----------------------------------------------------------------------
SUMMARY: The Assistant Secretary is developing these standards pursuant
to the Office of Educational Research and Improvement's authorizing
legislation, the ``Educational Research, Development, Dissemination,
and Improvement Act of 1994.'' The major purpose of these standards is
to provide quality assurance that programs designated by the Department
of Education as either exemplary or promising have met criteria that
will allow educators, professional organizations, and others to use
these programs with confidence.
DATES: Comments must be received on or before August 2, 1996.
ADDRESSES: All comments concerning these proposed regulations should be
addressed to Eve M. Bither, U.S. Department of Education, 555 New
Jersey Avenue, N.W., Room 500, Washington, D.C. 20208-5530. Comments
may also be sent through the Internet to: (Eve__Bither@ed.gov).
Comments that concern information collection requirements should be
sent to the Office of Management and Budget at the address listed in
the Paperwork Reduction Act section of this preamble.
FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt, U.S. Department of
Education, 555 New Jersey Avenue, N.W., Room 508, Washington, D.C.
20208-5643. Telephone: (202) 219-2126. Internet:
(Sharon__Bobbitt@ed.gov).
Individuals who use a telecommunications device for the deaf (TDD)
may call the Federal Information Relay Service (FIRS) at 1-800-877-8339
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.
SUPPLEMENTARY INFORMATION:
Background
On March 31, 1994, President Clinton signed Pub. L. 103-227, which
includes Title IX, the ``Educational Research, Development,
Dissemination, and Improvement Act of 1994'' (the ``Act''). The Act
restructured the Office of Educational Research and Improvement (OERI)
and provided it with a broad mandate to conduct an array of research,
development, dissemination, and improvement activities aimed at
strengthening the education of all students.
Statutory Requirements
The legislation directed the Assistant Secretary to develop, in
consultation with the National Educational Research Policy and
Priorities Board, such standards as may be necessary to govern the
conduct and evaluation of all research, development, and dissemination
activities carried out by the Office to ensure that those activities
meet the highest standards of professional excellence. The legislation
requires that the standards be developed in three phases.
In the first phase, standards were promulgated to establish the
peer review process and evaluation criteria to be used for the review
of applications for grants and cooperative agreements and proposals for
contracts. The final regulations setting out these standards were
published in the Federal Register on September 14, 1995 (60 FR 47808).
These proposed regulations address the second phase of development by
establishing the criteria to be used in reviewing potentially exemplary
and promising educational programs. The Assistant Secretary will
publish at a later date additional proposed regulations for phase three
of the standards, which are to govern evaluation of the performance of
recipients of grants and contracts and cooperative agreements with
OERI.
The OERI legislation requires that expert panels be established to
review educational programs and recommend to the Secretary those
programs that should be designated as exemplary or promising and
disseminated through the Department's National Education Dissemination
System. The legislation further requires the Assistant Secretary to
develop standards that describe the procedures the panels will use in
reviewing the educational programs. Section 941(a)(3) of the
legislation broadly defines ``educational program'' to include
``educational polices, research findings, practices and products.''
Educational programs may range in size and complexity from an
individual instructional program--such as an elementary school science
program--to a comprehensive reform initiative involving multiple goals
and participants. Programs at all levels of education--preschool,
elementary, secondary, and postsecondary--are eligible for
consideration.
The Act also requires that the Assistant Secretary review the
procedures utilized by the National Institutes of Health (NIH), the
National Science Foundation (NSF), and other Federal departments or
agencies engaged in research and development and actively solicit
recommendations from research organizations and members of the general
public.
In developing the review and evaluation procedures for the proposed
standards, OERI has reviewed and considered dissemination practices and
procedures used for identifying promising and exemplary programs by
various foundations, research organizations, associations, and Federal
agencies including NIH, NSF, the Federal Emergency Management Agency,
the Department of Health and Human Services, and the National Endowment
for the Arts. OERI adapted these review and evaluation procedures as
appropriate.
Proposed Standards
The proposed standards have been developed by the Assistant
Secretary in consultation with the Board. The standards proposed in
this NPRM--
* Require that expert panels be established to review
educational programs and recommend to the Secretary those programs that
should be designated as exemplary or promising and disseminated through
the Department's National Education Dissemination System; and
* Establish a process that panels will use to review and
evaluate educational programs and determine which programs to recommend
to the Secretary for designation as exemplary or promising.
Educational programs may be submitted at any time for consideration
for designation as exemplary or promising. In addition, the Assistant
Secretary will periodically establish and announce in the Federal
Register specific topic areas of high priority for which programs will
be invited or sought out. The legislation also provides that the
Secretary may identify educational programs for the panels to review.
Educational program submissions may include, as evidence of the
effectiveness of the program, a range of assessments, evaluative
information from users, and other objective performance indicators that
are appropriate to the program. The legislation ensures that a panel
may not eliminate any program from consideration based on the lack of
one type of supporting data such as test scores.
[[Page 27991]]
A standing group of experts, which will include teachers and
others, will be appointed by the Assistant Secretary as appropriate.
From that group, the Assistant Secretary will select members who have
relevant knowledge and experience in specific topic areas to form
expert panels to review programs in accordance with the criteria in
these proposed regulations.
In determining whether an educational program should be recommended
as exemplary or promising, the panel is required by the legislation to
consider (a) whether, based on empirical data, the program is effective
and should be designated as exemplary, or (b) whether there is
sufficient evidence to demonstrate that the program shows promise for
improving student achievement and should be designated as promising.
These proposed regulations require a panel to evaluate whether a
program has met all of the criteria of educational effectiveness set
forth in Subpart C of these proposed regulations. A panel may determine
that a program shows promise for improving student achievement and
recommend that the program be designated as promising if the program
has met all of the criteria with respect to one context, or with one
population. A panel may determine that a program is effective and
recommend that the program be designated as exemplary if the program
has met all of the criteria with respect to multiple contexts, or with
multiple populations.
Use of these criteria for evaluating programs will ensure that
programs disseminated by the Department are high-quality, research-
based programs that have provided evidence indicating they have
improved teaching or learning or both. The Department's dissemination
system is designed to make programs available to the public as quickly
as possible. The system will enable the Department to respond to all
forms of requests for information and assistance, and to support the
applications of research and best practice. The system will use
electronic networking and the capabilities of:
National Research Institutes;
Educational Resources Information Center (ERIC);
Regional Educational Laboratories;
Department-supported technical assistance providers;
National Library of Education; and
Other public and private nonprofit entities, including education
associations and networks.
Prior to the adoption of these standards, exemplary programs were
validated by the Department's Program Effectiveness Panel (PEP) and
disseminated through the National Diffusion Network (NDN). With the
adoption of these standards, the Department will recognize and
disseminate promising educational programs in addition to exemplary
programs.
Regulatory Flexibility Act Certification
The Secretary certifies that these proposed regulations would not
have a significant economic impact on a substantial number of small
entities.
The small entities that would be affected by these proposed
regulations are small local educational agencies (LEAs) and private
schools receiving Federal funds under this program. However, the
regulations would not have a significant economic impact on the small
LEAs and private schools affected, because the regulations would not
impose excessive regulatory burdens or require unnecessary Federal
supervision. The regulations would impose minimal requirements to
ensure the proper expenditure of program funds.
Paperwork Reduction Act of 1995
Section 701.4 contains information collection requirements. As
required by the Paperwork Reduction Act of 1995, the Department of
Education will submit a copy of this section to the Office of
Management and Budget (OMB) for its review. (44 U.S.C. 3504(h))
These regulations affect the following types of entities eligible
to submit a program for review: Any public or private agency,
organization or institution, or individual.
The public reporting burden is estimated to range from 2 to 6 hours
for each program submitted for review. The actual burden will be
determined by how much descriptive information about their program each
entity wishes to provide.
Organizations and individuals desiring to submit comments on the
information collection requirements should direct them to the Office of
Information and Regulatory Affairs, OMB, Room 10235, New Executive
Office Building, Washington, D.C. 20503; Attention: Wendy Taylor.
The Department considers comments by the public on this proposed
collection of information in--
* Evaluating whether the proposed collection of information
is necessary for the proper performance of the functions of the
Department, including whether the information will have practical
utility;
* Evaluating the accuracy of the Department's estimate of
the burden of the proposed collection of information, including the
validity of the methodology and assumptions used;
* Enhancing the quality, usefulness, and clarity of the
information to be collected; and
* Minimizing the burden of the collection of information on
those who are to respond, including through the use of appropriate
automated, electronic, mechanical, or other technological collection
techniques or other forms of information technology (e.g., permitting
electronic submission of responses).
OMB is required to make a decision concerning the collection of
information contained in these proposed regulations between 30 and 60
days after publication of this document in the Federal Register.
Therefore, a comment to OMB is best assured of having its full effect
if OMB receives it within 30 days of publication. This does not affect
the deadline for the public to comment to the Department on the
proposed regulations.
Invitation To Comment
Interested persons are invited to submit comments and
recommendations regarding these proposed regulations.
All comments submitted in response to these proposed regulations
will be available for public inspection, during and after the comment
period, in Room 600, 555 New Jersey Avenue, NW., Washington, DC,
between the hours of 8:30 a.m. and 4 p.m., Monday through Friday of
each week except Federal holidays.
Assessment of Educational Impact
The Secretary particularly requests comments on whether the
proposed regulations in this document would require transmission of
information that is being gathered by or is available from any other
agency or authority of the United States.
List of Subjects in 34 CFR Part 701
Education, Educational research, Reporting and recordkeeping
requirements.
Dated: May 22, 1996.
(Catalog of Federal Domestic Assistance Number does not apply)
Sharon P. Robinson,
Assistant Secretary for Educational Research and Improvement.
The Secretary proposes to amend Chapter VII of Title 34 of the Code
of Federal Regulations by adding a new Part 701 to read as follows:
[[Page 27992]]
PART 701--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES
CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT
(OERI)--DESIGNATION OF EXEMPLARY AND PROMISING PROGRAMS
Subpart A--General
Sec.
701.1 What is the purpose of these standards?
701.2 What definitions apply?
701.3 What entity is eligible to submit a program for review?
701.4 What must an entity submit for review?
Subpart B--Selection of Panel Members
701.10 How are panels established?
701.11 Who may serve as a member of the standing group?
701.12 How is the membership of expert panels determined?
Subpart C--The Expert Panel Review Process
701.20 How does an expert panel evaluate programs?
701.21 What is the difference between an exemplary and a promising
program?
701.22 What criteria are used to evaluate programs for exemplary or
promising designation?
Authority: 20 U.S.C. 6011(i)
Subpart A--General
Sec. 701.1 What is the purpose of these standards?
(a) The standards in this part implement section 941(d) of the
Educational Research, Development, Dissemination, and Improvement Act
of 1994.
(b) These standards are intended to provide quality assurance that
programs designated by the Department of Education as either exemplary
or promising have met criteria that will allow educators, professional
organizations, and others to use these programs with confidence.
(Authority: 20 U.S.C. 6011(i)(2)(E))
Sec. 701.2 What definitions apply?
Definition in the Educational Research, Development, Dissemination,
and Improvement Act of 1994. The following term used in this part is
defined in 20 U.S.C. 6041(a)(3):
Educational program
(Authority: 20 U.S.C. 6041(a)(3))
Sec. 701.3 What entity is eligible to submit a program for review?
Any public or private agency, organization, or institution, or an
individual, may submit an educational program for review.
(Authority: 20 U.S.C. 6011(i)(2)(E))
Sec. 701.4 What must an entity submit for review?
(a) To have its educational program considered for designation as
exemplary or promising, the eligible entity must submit to the
Secretary a description of the program and a discussion of the
program's educational effectiveness, responsive to the criteria in
Subpart C, Sec. 701.22.
(b) Information submitted must include, to the extent relevant to
the particular program--
(1) A program abstract of 250 words or less;
(2) A description of the salient features of the program;
(3) A description of the program's philosophy and history;
(4) Site information, including demographics;
(5) A description of evaluation results;
(6) Funding and staffing information; and
(7) Organization name, address, telephone and fax numbers, e-mail
address (if available), and contact person.
(Authority: 20 U.S.C. 6011(i)(2)(E))
Subpart B--Selection of Panel Members
Sec. 701.10 How are panels established?
(a) The Assistant Secretary, in consultation with the National
Educational Research Policy and Priorities Board established under 20
U.S.C. 6021, establishes a standing group of educational experts. The
Assistant Secretary may expand the membership of the standing group as
necessary.
(b) The Assistant Secretary selects members from the standing
group, based on their areas of expertise, to serve on expert panels in
specific topic areas for the purpose of reviewing and evaluating
educational programs and recommending, to the Secretary, those programs
that should be designated as exemplary or promising.
(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))
Sec. 701.11 Who may serve as a member of the standing group?
An individual may serve as a member of the standing group for the
purpose of reviewing and evaluating educational programs for exemplary
or promising status if that individual possesses two or more of the
following qualifications:
(a) Demonstrated expertise and experience in one or more specific
educational areas.
(b) Demonstrated expertise and experience across a broad range of
educational policies and practices.
(c) Experience in evaluating educational programs.
(d) Experience or expertise in developing educational products.
(e) Current employment as a teacher, principal or other school-
based or community-based professional (such as a guidance counselor,
school media specialist, or health professional).
(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))
Sec. 701.12 How is the membership of expert panels determined?
(a) For the review of each program, or group of programs, the
Assistant Secretary establishes an expert panel comprised of
individuals who are members of the standing group.
(b) In establishing the membership of each expert panel, the
Assistant Secretary--
(1) Selects individuals who have in-depth knowledge of the subject
area or content of the program or group of programs to be evaluated;
(2) Selects at least one current teacher, principal, or other
school-based or community-based professional;
(3) Ensures that no more than one-third of the panel members are
employees of the Federal Government; and
(4) Ensures that each panel member does not have a conflict of
interest, as determined in accordance with paragraph (c) of this
section, with respect to any educational program the panel member is
asked to review.
(c) Panel members are considered employees of the Department for
the purposes of conflicts of interest analysis and are subject to the
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's
policies used to implement those provisions.
(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))
Subpart C--The Expert Panel Review Process
Sec. 701.20 How does an expert panel evaluate programs?
(a) Each panel member shall--
(1) Independently review each program based on the criteria in
Sec. 701.22;
(2) Provide written comments based on an analysis of the strengths
and weaknesses of the program according to the criteria;
(3) Participate in site visits if appropriate; and
(4) Participate in a meeting of the expert panel, if appropriate,
to discuss the reviews.
(b) A panel may not eliminate an educational program from
consideration based solely on the fact that the program does not have
one specific type of supporting data, such as test scores.
[[Page 27993]]
(c) Each expert panel shall make a recommendation to the Secretary
as to whether the program is exemplary, promising, or neither.
(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d))
Sec. 701.21 What is the difference between an exemplary and a
promising program?
(a) A panel may recommend to the Secretary that a program be
designated as promising if the panel determines that the program has
met each of the criteria of educational effectiveness in Sec. 701.22
with respect to one context or one population.
(b) A panel may recommend to the Secretary that a program be
designated as exemplary if the panel determines that the program has
met each of the criteria of educational effectiveness in Sec. 701.22
with respect to multiple contexts or multiple populations.
(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d)(2))
Sec. 701.22 What criteria are used to evaluate programs for exemplary
or promising designation?
In determining whether an educational program (``program'' includes
educational polices, research findings, practices and products) should
be recommended as exemplary, promising, or neither, each expert panel
shall consider the following criteria of educational effectiveness:
(a) Evidence of success. The expert panel considers--
(1) Whether, based on a range of assessments, information from
users, or other indicators as appropriate, the program contributes to
solving substantial or important problems in teaching or learning; and
(2) The extent to which--
(i) Program effects are beneficial to the populations for whom the
program was designed; or
(ii) The product performs as expected for the educational consumers
it was said to benefit.
(b) Quality of the program. The expert panel considers--
(1) Whether the program has clear goals, is based on sound research
and practice, and incorporates accurate and up-to-date content;
(2) Whether the program represents a substantially improved
alternative to existing options;
(3) The extent to which the program promotes equity and is free of
bias based on race, gender, age, culture, ethnic origin, disability, or
limited English proficiency status;
(4) Whether the program is based on high expectations for the
success of all participants;
(5) Whether the program is appropriate to the target audiences; and
(6) The extent to which any materials associated with the program
conform to accepted standards of technical quality.
(c) Educational significance. The expert panel considers--
(1) The extent to which the program has the potential to increase
knowledge or understanding of educational problems, and issues, or
effective strategies for teaching or learning; and
(2) Whether the program is described clearly enough so that it can
be adapted or adopted in new sites.
(d) Usefulness to others. The expert panel considers--
(1) Whether the cost of the program (including money, staff time,
and other required resources) is reasonable in light of expected
benefits and compared to other alternatives; and
(2) Whether the program is available for use by others.
(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)(2))
[FR Doc. 96-13801 Filed 5-31-96; 8:45 am]
BILLING CODE 4000-01-P