Comparing click-through data to purchase decisions for retrieval evaluation

Authors
Publication date 2010
Host editors
  • H.-H. Chen
  • E.N. Efthimiadis
  • J. Savoy
  • F. Crestani
  • S. Marchand-Millet
Book title SIGIR 2010: proceedings: 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval: Geneva, Switzerland, July 19-23, 2010
ISBN
  • 9781450301534
Event 33rd Annual International ACM SIGIR Conference (SIGIR 2010), Geneva, Switzerland
Pages (from-to) 761-762
Publisher New York, NY: Association for Computing Machinery
Organisations
  • Faculty of Science (FNWI) - Informatics Institute (IVI)
Abstract Traditional retrieval evaluation uses explicit relevance judgments which are expensive to collect. Relevance assessments inferred from implicit feedback such as click-through data can be collected inexpensively, but may be less reliable. We compare assessments derived from click-through data to another source of implicit feedback that we assume to be highly indicative of relevance: purchase decisions. Evaluating retrieval runs based on a log of an audio-visual archive, we find agreement between system rankings and purchase decisions to be surprisingly high.
Document type Conference contribution
Language English
Published at
https://doi.org/10.1145/1835449.1835603 (Final published version)
Permalink to this page
Back