Generative models for pedestrian track analysis

Various problems in tracking and track analysis are addressed, with a focus on applications in the surveillance and intelligent vehicle domains, such as pedestrian path prediction, learning spatial and temporal structure of behavior patterns in data, anomalous track detection, and data association with inaccurate detections. It is shown that these low-level (that directly deal with the data) and high-level tasks (that deal with more abstract concepts) can be described by generative models which associate observations to one of several object labels (e.g. to construct tracks of objects in videos), or to one of several dynamics (e.g. to categorize dynamics within the tracks). These generative models are modular and adaptable to various problem settings, and provide a strong and unified theoretic foundation.
The Switching Linear Dynamical System (SLDS) can represent alternating pedestrian dynamics which transitions between walking and standing states. For online pedestrian path prediction from a moving vehicle, the SLDS is first extended with latent variables to exploit contextual information on where and when motion state transitions occur. Then, unsupervised learning of dynamics together with spatial context from unlabeled track data is considered. A spatial likelihood term is directly added to the SLDS states, such that these now capture dynamics for a specific spatial regions. This unsupervised approach is further extended into a hierarchical model to additionally cluster tracks into behavior classes.
Finally, these unsupervised techniques are applied to the data association problem in tracking itself, focusing on anomalous surveillance situations where object detections are inaccurate and motion models unreliable.