5-15


subsets as the degree of association (defined by the matching relation)

is decreased.  ~ach of these techniques is commonly used to represent

the variation in the joint distribution of the user/system decisions as

the quantity of output (the size of the retrieved subset) is increased

(or equivalently as the matching criterion is relaxed).   in section 4

of this chapter an alternative to the general evaluation strategy of

describing performance by a set of parameters which vary with discrete

changes in the matching criterion is presented.


    D.  ~he Precision-Recall ~radeoff

        ~he use of a precision vs. recall plot variable with the

cutoff parameter as an evaluation tool for document retrieval systems
                        6
(introduced by Cleverdon ) has led to observations that there exists a

so-called tradeoff between *these two conditional probabilities which

is of fundamental significance.  It will be shown here, however, that

this inverse relationship is a direct consequence of assuming a

statistically significant matching function, and further that both

of these conditional probabilities are increased by any process which

im~roves the joint probability of retrieval and relevance.

        ~he increase in recall as the amount 9f output accepted as

retrieved is increased i~ a direct consequence of the definition of

the recall conditional probability.  Since the retrieved subset is

monotonically increasing, the ratio of relevant documents retrieved to

tdtal number of relevant documents (a constant for any retrieval

operatidn) is necessarily monotonically increasing.  Precision,'

however, is defined as the ratio of relevant documents retrieved to