Vulnerable road user detection and orientation estimation for context-aware automated driving

This thesis addresses the detection, segmentation and orientation estimation of persons in visual data. In particular, the aim of this work is to establish an accurate machine representation of the Vulnerable Road Users (VRU, e.g. pedestrians, cyclists) by using image-based cues to support context-aware automated driving.
A robust detection of the VRU is achieved by applying efficient stereo-based proposals within region-based Convolutional Neural Networks. Various network and proposal configurations are compared on a newly introduced dataset focusing on the challenging detection of cyclists in urban areas.
A pixel-wise segmentation of the detected VRU facilitates higher-level, semantic scene analysis (e.g. pose estimation, activity analysis). Accurate object segmentations are gained by combining statistical shape models with multiple visual data cues within an iterative framework using a Conditional Random Field formulation.
Head and body part locations and orientations are jointly estimated from a set of orientation-specific detector responses. The applied Dynamic Bayesian Network model accounts for spatial and temporal anatomical constraints resulting in stable part localization and orientation estimates.
The inferred orientations are used to anticipate the behavior of the VRU by modeling situational awareness within a context-based Switching Linear Dynamic System. Experiments show that such context-aware models lead to a significant improvement in VRU path prediction.
Since data annotation and management are indispensable components for the development of complex machine learning applications, two software tools are proposed to support an efficient handling of sensor data and annotations.