On the topical structure of the relevance feedback set

We investigate the topical structure of the set of documents used to expand a query in pseudo-relevance feedback (PRF). We propose a coherence score to measure the relative topical diversity/compactness of this document set, which we call the relevance feedback set. The coherence score captures both the topical relatedness of the members of a document set as well as the set's overall clustering structure. We demonstrate the ability of the coherence score to capture topical structure of a document set using tests performed on synthetic data. Then, experiments are performed which show that when queries are divided into two sets, one with high and one with low relevance feedback set coherence, two different levels of retrieval performance are observed. In particular, our results suggests that if a query has a relevance feedback set with a high coherence score, it is apt to benefit from PRF.