meIRL-BC: Predicting Player Positions in Video Games

In this paper we demonstrate how behaviour-classification models can improve player position prediction for video game AI. To this end, we propose a novel method named meIRL-BC, which (1) uses maximum-entropy Inverse Reinforcement Learning for the creation of position prediction models [15], and (2) predicts player positions based on estimates of their most likely behavioural roles in the game (e.g., attacking, defending, ambushing, etc.). Experiments that test meIRL-BC in an actual Capture the Flag video game yielded the following three results. First, depending on the behavioural role to be classified, the prediction accuracy of meIRL-BC approximates, slightly improves, or substantially improves upon that of meIRL. Second, the accuracy of the trained behavioural classifier is presently insufficient for classifying actual game-player behaviour. Third, surprisingly, despite use of an ineffective classifier, both meIRL-BC and meIRL yielded comparable overall performance in actual gameplay. What is more, when in actual gameplay the percentage of correctly classified instances was above approximately 40%, meIRL-BC substantially outperformed meIRL is terms of win/loss/draw ratio, the number of bots that were successfully killed, and the absolute number of flags that were captured. Also, of all AIs in competition with the standard AISandbox AI, meIRL-BC was able to capture opponent flags more rapidly, thereby generally achieving a higher game score than meIRL. From these results, we can draw the conclusion that when the meIRL-BC method is correctly applied in video-game AI, it indeed outperforms meIRL in actual gameplay.