Herded Gibbs Sampling

Open Access
Authors
  • L. Bornn
  • Y. Chen
  • N. de Freitas
  • M. Eskelin
Publication date 2013
Book title International Conference on Learning Representation 2013
Event 1st International Conference on Learning Representations: ICLR 2013, in conjunction with AISTATS 2013
Publisher Ithaca, NY: ArXiv
Organisations
  • Faculty of Science (FNWI) - Informatics Institute (IVI)
Abstract
The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an O(1/T) convergence rate for models with independent variables and for fully connected probabilistic graphical models. Herded Gibbs is shown to outperform Gibbs in the tasks of image denoising with MRFs and named entity recognition with CRFs. However, the convergence for herded Gibbs for sparsely connected probabilistic graphical models is still an open problem.
Document type Conference contribution
Note All conference submissons on arXiv; accepted paper
Language English
Published at http://arxiv.org/abs/1301.4168
Downloads
1301.4168.pd (Submitted manuscript)
Permalink to this page
Back