learning parameter with missing values

The front end.
Post Reply
heyifan
Posts: 7
Joined: Tue Aug 25, 2020 2:05 pm

learning parameter with missing values

Post by heyifan »

I learnt that it is better to use 'Randomize' option when learning parameter with missing values. The problem is that using different random seeds leads to different learning results. How should I know which one is the optimized learning results? Maybe it is due to the size of training dataset is too small so the learning result doesn't converge? I really appreciate it if someone could help.
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: learning parameter with missing values

Post by marek [BayesFusion] »

Hi HeyFan,

Uniform distributions are as good as (or as bad as) a randomly chosen starting point and you should not view them as the best! To the contrary, it often leads to worse convergence. If you are worried about different convergence points, I suggest that you try a few randomizations and compare the log-likelihood of the resulting models. The higher the log-likelihood (it is negative, so this means the lowest absolute value!), the better the quality of the model. While all parametrizations should be close, the one with the highest log-likelihood should be the one that you want.
I hope this helps.

Marek
heyifan
Posts: 7
Joined: Tue Aug 25, 2020 2:05 pm

Re: learning parameter with missing values

Post by heyifan »

marek [BayesFusion] wrote: Mon Nov 29, 2021 3:24 pm Hi HeyFan,

Uniform distributions are as good as (or as bad as) a randomly chosen starting point and you should not view them as the best! To the contrary, it often leads to worse convergence. If you are worried about different convergence points, I suggest that you try a few randomizations and compare the log-likelihood of the resulting models. The higher the log-likelihood (it is negative, so this means the lowest absolute value!), the better the quality of the model. While all parametrizations should be close, the one with the highest log-likelihood should be the one that you want.
I hope this helps.

Marek
Thanks for your help. Another question about parameter learning. In GeNIe mannual Version 3.0.R2 section 6.5.8, data file Credit10K.csv is used to learn parameter of model Credit.xdsl. My question is that the data file doesn't contain missing values, why still use EM algorithm not MLE(Maximum Likelyhood Estimate)?
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: learning parameter with missing values

Post by marek [BayesFusion] »

No missing values is just a special case for the EM algorithm, so there is no need for a special separate algorithm for data with no missing values. I hope this helps,

Marek
Post Reply