Unsupervised (parameter) learning for MRFs on bipartite graphs

Tomas Sixta (CTU Prague, Czech Republic)

Abstract:

We consider unsupervised (parameter) learning for general Markov random fields on bipartite graphs. This model class includes Restricted Boltzmann Machines. We show that besides the stochastic gradient approximation (a.k.a. persistent contrastive divergence) there is an alternative learning approach – a modified EM algorithm which is tractable because of the bipartiteness of the model graph. We compare the resulting double loop algorithm and the persistent contrastive divergence learning experimentally and show that the former converges faster and more stable than the latter.