This was a first for the agency and was directly attributable to the efforts of the Patricia Harris-Thomas, Director of HRM's Personnel Research and Assessment Division, and her staff of industrial psychologists.

Their efforts paid off. According to a recent survey of chief inspectors, the QR hires who were screened via written testing and structured oral interview examinations show a better ability to grasp written information, and they bring a greater expertise to the job.

The Personnel Research and Assessment Division (PRAD) continues to develop and oversee a rapid expansion of testing at Customs, including testing for merit promotion through CareerFinder.

Harris-Thomas shares with readers her perspective on the testing process:

Customs began testing QR applicants several years ago - what can you tell us about their performance academically and on the job?
We've looked closely at the data from QR candidates' the entry-level inspectors and canine enforcement officers - since testing began.

We find that test scores, in particular the verbal/writing/logical reasoning area of the exam are very good overall predictors of academic success at the U.S. Customs Service Academy. This is because the tests were carefully developed to measure the competencies that new hires must have for effective performance at the Academy and on the job.

The academic scores of employees who came in through the QR test process are better overall than the scores from previous year's classes. In addition, new QR hires repeat fewer Academy classes.

In a recent survey, supervisory inspectors reported that QR hires are eager to learn, motivated, cooperative, and dedicated. These results are significant—they establish that testing helps to improve the quality of the Customs workforce.

Chief inspectors were generally satisfied with QR hires, but they recommended that Customs do a better job of retaining good QR recruits.

What are your goals when developing tests, such as the QR entrance examination or merit promotion examinations?

The ultimate goal is to predict performance, but more specifically, there are two critical testing goals.

First, when we develop and administer a test, we want to maintain the quality of the testing instrument, and ensure the examinations identify quality applicants and promotion eligibles.

Our second goal is to ensure that the test is fair. Professional test development standards require that special care be taken to ensure that the examination system is job related. The test must be highly objective and directly relevant to the mission.

What abilities and skills do the examinations measure?
The tests are designed to evaluate logical reasoning abilities, writing skills, interpersonal skills, and other competencies that are directly related to the job. For example:

• Entry-level inspector and agent examinations focus on quantitative and verbal reasoning abilities.
• Promotion examinations for special agent GS-14 and supervisory inspector GS-12 positions include specific job knowledge questions to assess the quality of technical experience, combined with a critical thinking/reasoning skills test, and an in-basket assessment of supervisory and administrative skills.
• Other exams, such as the Management Development Program (MDP), do not assess specific job knowledge, but focus instead on general managerial thinking, writing, and administrative skills.

A screening panel evaluated and ranked the overall records of applicants who competed under the Customs Candidate Development Program for Senior Executive Service developmental positions.

The best-qualified applicants then undergo an examination consisting of a group exercise, an oral presentation, and an experience-based interview. These examinations are designed to measure interpersonal, communication, problem solving, and leadership skills.

The testing process also provides supervisory/managerial participants the opportunity to evaluate their strengths and weaknesses. In addition to the Notice of Results from OPM, PRAD provides each participant with a breakout of his or her score as compared with the scores of others who took the test.

Photo Credit: Gerald L. Nino

Customs Personnel Research and Assessment Division Director Patricia Harris-Thomas has served as a personnel research psychologist for 30 years and has managed testing functions for the Office of Personnel Management and the Immigration and Naturalization Service before joining Customs in 1998. Harris has also taught statistics and psychological testing during her graduate studies and served as guest lecturer for local universities.

How does Customs CareerFinder relate to the agency's testing initiatives?
Customs CareerFinder (Intranet at http://customsnet and Internet at http://www.customs.gov) is gradually replacing existing merit promotion and external hiring processes. Applicants for supervisory and managerial positions such as MDP, SA-14, and supervisory inspector GS-12 use CareerFinder to sign up for the required examinations. The examinations are then scheduled and administered using traditional paper/pencil formats.

Applicants for non-supervisory positions under CareerFinder are not required to take a test. Instead, they use CareerFinder to complete an on-line application, which includes specific job-related questions developed by the PRAD staff.

Candidates' answers enable automatic qualification, rating, and ranking. As a result, no further evaluation is required.

What's on the horizon for testing at Customs?
PRAD plans to conduct seminars at various Customs locations to coach participants on how to improve their scores on promotion tests. A future goal is to move toward online testing which could allow test candidates to receive their scores immediately following an exam. We're considering working with commercial testing facilities to offer this service.

Video-based testing is also a future possibility for entry-level inspector positions. This type of test will ask participants to respond to virtual, videotaped situations similar to those they'll encounter in the field.

The history of testing at Customs Prior to the inception of Customs Quality Recruitment program (QR) in 1998, a variety of methods were used to identify the best qualified candidates for Customs positions such as generic tests administered by the Office of Personnel Management, and knowledge, skills, and abilities (KSAs) essays evaluated by subject matter experts. Beginning with entry-level inspector and canine enforcement officer positions in 1998, to identify the best qualified candidates, the agency began to phase in Customs-specific testing for a growing number of positions (see below): 1998 Entry-level inspectors and canine enforcement officers 1999 Entry-level special agents Customs pilots and aviation enforcement officers 2000 Special agent GS-14 2002 CareerFinder: GS-0080 Personnel Security Specialist GS-0132 Intelligence Research Specialist GS-0303 Management Program Technician GS-0318 Secretary GS-0326 Office Automation Clerk/Assistant GS-0343 Management Analyst GS-0950 Paralegal Specialist GS-1802 Investigative Assistant GS-1811 Special Agent (GS-12) GS-1811 Special Agent (GS-14) GS-1889 Import Specialist GS-1890 Customs Inspector (Summer and Veterans Readjustment Appointments) GS-1890 Supervisory Customs Inspector (GS-12) GS-1894 Entry Specialist GS-1897 Customs Technician GS (various) Management Development Program (MDP)