EM algorithm for discrete model in GeNIe

The engine.
Post Reply
snowave
Posts: 22
Joined: Mon Jan 25, 2016 1:27 pm

EM algorithm for discrete model in GeNIe

Post by snowave »

Hi, it seems most materials available online are explaining the EM for GMM models, i.e. generate expected sufficient statistics for Gaussians.
But how EM works on discrete models? Especially the algorithm used in GeNIe? Can I ask for a reference or simple explanations? Thanks a lot.

Peng
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: EM algorithm for discrete model in GeNIe

Post by marek [BayesFusion] »

There is plenty literature available on the topic but, roughly speaking, the EM algorithm consists of replacing successively the missing value in the data set based on the posterior probability of the various states (e.g., you pick the most likely value). Once the values have been replaced, you can learn the parameters and then establish the new best values. This is being done in a loop until the changes in the parameters are sufficiently small (below the a-priori established threshold). Does this help?
Cheers,

Marek
snowave
Posts: 22
Joined: Mon Jan 25, 2016 1:27 pm

Re: EM algorithm for discrete model in GeNIe

Post by snowave »

Thanks, so for the discrete EM we are actually filling in the missing data with expected values and then use the complete data to learn parameters.

Peng
Post Reply