Previous Research: 1999-2001

Task 2: Develop technical refinements to and conduct longitudinal validations of prototype modular, computer-based selection tests

Sub-task 2.1: Integrate Variable Item Generator (VIGOR) technology with AF/BEST for on-line generation of test items

The Airway Facilities Basic Electronics Screening Tool (AF/BEST) was a computer-administered test of electronics and applied mathematics knowledge for the Electronics Technician (GS-856), Environmental Technician (GS-802), and Airway Transportation Systems Specialist (GS-2101) occupations. AF/BEST was implemented in a single form, with no alternative versions available. To facilitate test security and integrity, alternate versions of the operational test were required.

Traditionally, alternate test questions are developed by subject matter experts and then empirically equated by administering them to large numbers of subjects. Development of alternate forms is, therefore, a lengthy, time-consuming, and expensive process. Software-based item generation technology could streamline the process of developing alternate versions of a test. The purpose of this sub-task was to develop a prototype computerized variable item generator (VIGOR) for the AF/BEST and evaluate the feasibility and development requirements associated with implementing a software-based item generator for an employment test.

This project involved conducting research to identify the components or attributes of an item that could be changed and the range of values that could be used for those components, without affecting the difficulty level of the item. Based on the results of the research, a software-based item generator was developed and integrated with the AF/BEST test software, and delivered to the FAA in May 2000. The research effort demonstrated the feasibility of computer generation of parallel test items.

Research Products