Inoculating Relevance Feedback Against Poison Pills

Relevance Feedback (RF) is a common approach for enriching queries, given a set of explicitly or implicitly judged documents to improve the performance of the retrieval. Although it has been shown that on average, the overall performance of retrieval will be improved after relevance feedback, for some topics, employing some relevant documents may decrease the average precision of the initial run. This is mostly because the feedback document is partially relevant and contains off-topic terms which adding them to the query as expansion terms results in loosing the retrieval performance. These relevant documents that hurt the performance of retrieval after feedback are called “poison pills”. In this paper, we discuss the effect of poison pills on the relevance feedback and present significant words language models (SWLM) as an approach for estimating feedback model to tackle this problem. Significant words language models are family of models aiming to estimate models for a set of documents so that all, and only, the significant shared terms are captured in the models. This makes these models to be not only distinctive, but also supported by all the documents in the set. To do so, SWLM assumes that terms in the each document in the set are drawn from mixture of three models:
1. General model, representative of common observation, 2. Specific model, representative of partial observation, and 3. Significant Words model, latent model representing the significant characteristics of the whole set. Then, it tries to extract the significant words model.