Distributed Inference of LDA using Expectation-Propagation and GBR?

After learning expectation-propagation (EP) from PRML, I think it is possible to program EP algorithms designed for inference in factorized graphical models using GBR. However, the following issues are to be confirmed:

  • There has no proof that EP converges to a (local or global) optima. However, the EP algorithm developed for LDA [1] seems converges.
  • The EP for LDA [1] is in fact an extended EP algorithm, but not literally an EP algorithm. Need to make sure that this extended EP can be programmed using GBR.
  • Need to estimate the number and size of messages need to passed between workers. This is an approximate estimate of the communication cost of the GBR program.


  1. Thomas Minka and John Lafferty, Expectation-propagation for the generative aspect model, UAI 2002