SP500207
NIST Special Publication 500-207: The First Text REtrieval Conference (TREC-1)
Natural Language Processing in Large-Scale Text Retrieval Tasks
chapter
T. Strzalkowski
National Institute of Standards and Technology
Donna K. Harman
wordi word2 SIMnorm
abm
absence
accept
accord
acquire
speech
adjustable
maxsaver
affair
affordable
disease
medium+range
aircraft
aircraft
airline
alien
anniversary
anti+age
anti+clot
contra
candidate
contend
property
attempt
await
stealth
child
baggage
ban
bearish
bee
roller+coast
two +income
television
soldier
treasury
research
withdrawal
Table 2. Filtered
more specific term).
*anti+ballistic
*maternity
acquire
pact
purchase
address
one +year
*advance +purchase
scandal
low+income
*ailment
*air+to+air
*jetliner
plane
carrier
immigrate
*bicentennial
anti+wrinkle
cholesterol+lower
*anti+sandinista
*aspirant
*aspirant
asset
bid
pend
*b+1
*baby
luggage
restrict
bullish
*honeybee
*bumpy
two+earner
tv
troop
*short+term
study
*pullout
0.534894
0.233082
0.179078
0A92332
0A49362
0.263789
0.824053
0.734008
0.684877
0.181795
0.247382
0.874508
0.166777
0.423831
0.345490
0.270412
0.588210
0.153918
0.856712
0.294677
0.116025
0.143459
0.285299
0.641592
0.572960
0.877582
0.183064
0.607333
0.321943
0.847103
0.461023
0.898278
0.293104
0.806018
0.374410
0.661133
0.209257
0.622558
word similarifies (* indicates the
terproductive with larger databases. Later experiment have
confirmed this as we obtained new code from NIST.
183
wordi word2 SIMnorm
lord abuser 0.261507
break accelerator 0.106775
accept reject 0.275802
accord level 0.152023
cent accrue 0.259478
acquire deficient 0.615525
fix+rate adjustable 0.930180
advertise environmental 0.124026
petroleum aerospace 0.207406
afghan nicaraguan 0A21478
banana dominican 0.444193
begin month 0.346558
german british 0.112465
superpower reagan+ gorbachev 0.400145
republic sea 0.227662
sunglasses sneakers 0.126749
Table 3. Some (apparently?) "bad" similarities gen-
erated.
also quite disappointed with the effect that the query
expansion had (or did not have) on either precision or
recall figures. This was somewhat more directly
related to the wide domain scope of WSJ articles, and
we noticed many obvious word sense mixups leading
to unwanted term associations. Also, the rather
straightforward method of query expansion terms
proved less adequate here, and we are considering
new methods as outlined briefly in the introduction to
this paper. Our final test with manually altered
queries (terms were deleted or added after consulting
the original topic) brought only negative results: any
kind of intervention appeared only to further decrease
both recall and precision.
Final runs were performed as follows:
(1) Topics 1-25 were processed using only direct
fields (<title>, <desc>, and <narr>) with
respect to the training database (disk 1). They
were subsequently run in the routing mode
against the test database (disk 2). 26 Unfor-
tunately, the routing results were hindered by a
bug in our program that removed duplicate
26 While other fields provided much useful information
about various aspects of a given topic (including definitions of spe-
cialized terms), we thought that their inclusion in the final query
would make it difficult to assess the advantages of linguistic pro-
cessing. This especially applies to Concepts fiels <con> which list-
ed hand-extracted keywords. Subsequenfly, we realized that more
often this field in fact provided further clues not found in other sec-
tions of a topic.