Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i6TEbv120652; Thu, 29 Jul 2004 10:37:57 -0400 (EDT) Date: Thu, 29 Jul 2004 10:37:57 -0400 (EDT) Message-Id: <003c01c4757a$4692eb30$0502a8c0@frodo> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: "Marie Cora" <marie.cora@hotspurpartners.com> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:615] RE: what counts as legitimate outcomes X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas X-Mailer: Microsoft Outlook, Build 10.0.2627 Content-Transfer-Encoding: 7bit Content-Type: text/plain; Status: O Content-Length: 4192 Lines: 99 Hi George and everyone, Thanks for your report George - it's good to get a sense of what folks are using, and what they use for which purposes. Agreed: REALLY difficult to simultaneously measure program effectiveness and student literacy gain. So far, there are no instruments that can do both - which is why we are having the conversation about the challenge of meeting state and federal requirements, and of measuring student progress. Some assessments are built for the purpose of measuring gains in learning; others are built to determine efficacy of a program on a larger scale. Again herein lies the dilemma: we need to capture legitimate outcomes in student learning gains on an individual basis; we need to capture legitimate outcomes of the efficacy of the program. Both are legitimate activities - but for different reasons (purposes). I have a couple of clarifying statements/questions: George mentions a distinction between curriculum-based and (versus?) standardized tests. Remember that you can (theoretically) standardize any assessment - the point being that curriculum-based assessments can also be standardized. I think, George, by curriculum-based, you mean an assessment that is directly developed from the curriculum (an example would be, say, a skills textbook that has a 'chapter test' at the end of a unit = NOT standardized; or say, the assessment tool/process that goes along with the Read with Understanding portion of EFF = standardized.) George also mentions the DAR - but I'm not sure that this is a standardized test actually - I know there is only one version of this test, and so I wonder about reliability studies. Anyone out there shed light on the DAR and how it was developed? The Wilson Reading System, I believe was originally developed for use with learning disabilities - again, I do not know how this test was constructed, and so it's unclear if this is standardized - anyone have more info on this? Finally, George poses this question: should his program spend time collecting information on interim gains? Do other programs do this? If so, how? Thanks! marie cora Moderator, NIFL Assessment Discussion List, and Coordinator/Developer LINCS Assessment Special Collection at http://literacy.kent.edu/Midwest/assessment/ marie.cora@hotspurpartners.com -----Original Message----- From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On Behalf Of George Demetrion Sent: Tuesday, July 27, 2004 4:16 PM To: Multiple recipients of list Subject: [NIFL-ASSESSMENT:613] what counts as legitimate outcomes Colleagues, As with many programs, our agency also is dealing with what counts as legitimate outcomes, a particularly important matter when it becomes difficult to evaluate program efficacy through directly assessable measures. Our Basic Literacy program uses three formal instruments, although I think only two of them would be considered standardized. Those would be the CASAS, which meets our requirement with the state, and also provides us with usable information, and the Diagnostic Assessment of Reading (DAR) which we've instituted about a year ago and are just now beginning to obtain post-test scores. Part of our program consists of the Wilson Reading System (WRS) that includes unit or section tests. The WRS is a systematic phonics program and consists of 12 sequential levels. Each of the set of skills in each level builds on the skills of the preceding levels. Each of the levels has a number of sublevels which also are also sequentially based. Our top students are in Level 5. Up to now we have only included level-to-level gains as legitimate outcomes that we include in some of our reportable data. Last year we had 15 students out of about 150 with level gains. The question we are hashing over is whether it is legitimate (and worthwhile)to also count gains in sub-levels even if folks stay in the same level. The measurements (whether levels or sublevels) are curriculum-based rather than standardized, though results can be clearly quantified. Feedback particularly among those utilizing the WRS as well as from others will be appreciated. Thanks, George Demetrion
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:18 EST