11-1*7

          Relevance                             Relevance
  Rank      Grdde      Products        Rank       Grade     Products

    1         1+          14              1         1          1

    2         3           6               2         2          14

    3         2           6
                                          3         3          9
    14        1           14              14        14        16

   Sum of Products  20                  Sum of Products   30

~eighted Normed. Recall   1.000       Weighted Normed. Recall .9872


(a)  Perfect Ranks and Perfect        (b)  Perfect Ranks and Worst

     Relevance Grade Order                 Relevance Grade Order


          Relev rce                             Relevance
  Rank      Grade      Products        Rank       Grade     Products

    1         14          14              3         3          9

    3         3           9             13          2         26
    14        2           8             19          14        76
    9         1           9             141         2         82

   Sum of Products  30                  Sum of Products   193

Weighted Normed. Recall   .9872       Weighted Normed. Recall .78141*

(c)  Less than Perfect Ranks

     and Perfect Relevance

     Grade Order

(d)  Actual Performance of

     Cran-l Request ~67,

     Suffix 15t Dictionary

           Examples of Use of Weighted Normalized Recall.

                              Fig. 27.