SP500215 Design and Evaluation of the CLARIT-TREC-2 System chapter D. Evans R. Lefferts National Institute of Standards and Technology D. K. Harman C)fficiai TR8C-2 R[OCRerr]uuit [OCRerr]L[OCRerr]RI[OCRerr] [OCRerr] : [OCRerr][OCRerr]uting [OCRerr][OCRerr]rfe,rm[OCRerr]ric[OCRerr] 1 0.6 - 0.6 0.4 - 0.3 - 0 I 0.0 0.1 0.3 0.3 0.4 Q.6 0.6 0.7 0.8 0.9 1.0 R*oaii Automatia Routing [OCRerr] 1000 [OCRerr]oaummnta [OCRerr] Avaraga - .3269 [OCRerr]anuai Routing [OCRerr] 1000 Doaumanta [OCRerr] A[OCRerr]mraga - .3302 Figure 2: P-R Curve for Routing Over 1000 Docs[OCRerr]LARTM and CLARTA I [OCRerr]LAK[OCRerr][OCRerr]i M[OCRerr] K[OCRerr][OCRerr]ounng Q[OCRerr]-uenes, 5i-iuu ARIA R-outinR Q[OCRerr]ueries, 5-i[OCRerr]uu lotal number of documents Total number of documents over all queries over all queries Retrieved: 50000 Retrieved: 50000 Relevant 10489 Relevant 10489 Rel[OCRerr]t 6785 Rel[OCRerr]t 6811 Interpolated Recall-Precision Averaget: Interpolated Recall-Precision Averages: at 0.00 0.7791 at 0.00 0.8101 at 0.10 0.5825 at 0.10 0.5781 at 0.20 0A987 at 0.20 0A941 at 0.30 0A505 at0.30 0A368 at 0.40 0.3848 at 0.40 0.3851 at 0.50 0.3295 at 0.50 0.3358 at 0.60 0.2739 at 0.60 0.2749 at 0.70 0.2109 at 0.70 0.2074 at 0.80 0.1683 at 0.80 0.1561 at 0.90 0.09[OCRerr] at 0.90 0.0933 atl.00 0.0126 atl.00 0.0141 Average Precision (non-interpolated) Average Precision (non-interpolated) over all rel does: 0.3302 over all rel does: 0.3269 Precision: Precision: At 5 does: 0.6120 AtS does: 0.6200 At 10 does: 0.5940 At 10 does: 0.6060 At 15 does: 0.5800 At 15 does: 0.5893 At 20 does: 0.5700 At 20 does: 0.5730 At 30 does: 0.5527 At 30 does: 0.5460 At 100 does: 0A396 At 100 does: 0A346 At2OO docs: 0.3436 At 200 does: 0.3416 At5OO docs: 02176 At 500 does: 0.2184 At 1000 does: 0.1357 At 1000 does: 0.1362 R-Precision (precision after R retrieved) R-Precision (precision after R retrieved) Exact 0.3642 Exact 0.3646 Table 1: Offical Results, Routing Topics 51-100 141