Improving Sequential Recommenders through Counterfactual Augmentation of System Exposure

In sequential recommendation, system exposure refers to items that are exposed to the user. Typically, the user only interactions with a few of the exposed items. Although sequential recommendation has achieved great success in predicting future user interests, existing sequential recommendation methods do not fully exploit system exposure data. Most methods only model items that have been interacted with, while the large volume of exposed but non-interacted items is overlooked. Even methods that consider system exposure typically train the recommender using only the logged historical system exposure, without exploring unseen user interests.
In this paper, we propose counterfactual augmentation over system exposure for sequential recommendation (CaseRec). To better model historical system exposure, CaseRec introduces reinforcement learning to account for different exposure rewards. CaseRec uses a decision transformer-based sequential model to take an exposure sequence as input and assigns different rewards according to the user feedback. To explore unseen user interests, CaseRec performs counterfactual augmentation, where exposed items are replaced with counterfactual items. Then, a transformer based user simulator is used to predict the user feedback reward for the augmented items. Augmentation, together with the user simulator, gives rise to counterfactual exposure sequences to uncover new user interests. Finally, CaseRec uses the logged exposure sequences with the counterfactual exposure sequences to train a decision transformer-based sequential model for generating recommendation. Experiments on three real-world benchmarks show the effectiveness of CaseRec. Our code is available at https://github.com/ZiqiZhao1/CaseRec.