Objects for spatio-temporal activity recognition in videos

This thesis investigates the role of objects for the spatio-temporal recognition of activities in videos. More specifically, we investigate what, when, and where specific activities occur in visual content by examining object representations, centered around the main question: what do objects tell about the extent of activities in visual space and time? The thesis presents six works on this topic. First, the spatial extent of activities is investigated using objects and their parts. Second, over two works, it is investigated whether activities exhibit different object preferences over time and which objects matter for representing activities. Third, the full spatio-temporal extent of activities is investigated, where over three works the extensive annotation burden of action localization is replaced respectively with point annotations, pseudo-annotations, and a zero-shot setting, where no video examples are given during training. The works lead to the conclusion that objects provide valuable information about the presence and spatio-temporal extent of activities in videos.