ViTOR: Learning to Rank Webpages Based on Visual Features

The visual appearance of a webpage carries valuable informationabout page’s quality and can be used to improve the performanceof learning to rank (LTR). We introduce the Visual learning TO Rank(ViTOR) model that integrates state-of-the-art visual features extraction methods: (i) transfer learning from a pre-trained image classification model, and (ii) synthetic saliency heatmaps generated from webpage snapshots. Since there is currently no public dataset availablefor the task of LTR with visual features, we also introduce and releasethe ViTOR dataset, containing visually rich and diverse webpages.The ViTOR dataset consists of visual snapshots, non-visual featuresand relevance judgments for ClueWeb12 webpages and TREC WebTrack queries. We experiment with the proposed ViTOR model onthe newly introduced ViTOR dataset and show that our model significantly improves the performance of LTR with visual features.