Deep Belief Networks for dimensionality reduction

Deep Belief Networks are probabilistic generative models which are composed by multiple layers of latent stochastic variables. The top two layers have symmetric undirected connections, while the lower layers receive directed top-down connections from the layer above. The current state-of-the-art training method for DBNs is contrastive divergence, an efficient learning technique that can approximate and follow the gradient of the data likelihood with respect to the model parameters. In this work we explore the quality of the non-linear dimensionality reduction achieved through a DBN on face images. We compare the results achieved to the well know Principal Component Analysis as well as with a Harmonium model, which is the top layer of a DBN.