Setting Beta(Dirichlet) prior when using jsmile EM

The engine.
Post Reply
Yun
Posts: 20
Joined: Mon Oct 17, 2016 8:34 pm

Setting Beta(Dirichlet) prior when using jsmile EM

Post by Yun »

Dear SMILE creators,

Thank you so much for your time! Is there a way to specify Beta distribution prior by jsmile (when I am using EM to learn the parameters), and if there is, what is the right way to do that?

-- Based on the previous post http://support.bayesfusion.com/forum/vi ... prior#p688 which said "it's the equivalent sample size associated with the Dirichlet distribution which determines the prior over the parameters". However, I don't understand what it meant by "higher values will enforce more uniformly distributed CPTs" following the previous sentence: if this equivalent sample size equals to alpha+beta(i.e., a+b) for a beta distribution, wouldn't the CPT also depends on the relative value of alpha over beta (i.e., mean/mode of the Beta distribution)? E.g., even if we have very large equivalent sample size (i.e., high values), we can still have very big alpha relative to beta, resulting in a distribution biased to high probabilities rather than a uniform distribution?

-- Meanwhile, I tried to get the default equivalent sample size from my current jsmile code by "em.getEqSampleSize()", and it shows 1. Can I understand this value as follows: if the prior mean is 0.75, then alpha=0.75*1=0.75 and beta=1-0.75=0.25. Is this correct? So if I want to have a very strong prior, I could set equivalent sample size very large, such as 200? At the end if I have strong belief that the probability of a random variable X=T is around 0.75, then I should initialize the variable by "net.setNodeDefinition(X, new double[]{0.75, 0.25}", and then use "em.setEqSampleSize(200)". Is this correct?

-- It seems that the above way will set the equivalent sample size for all parameters at the same time. Is there a way to set it differently for different parameters?

Thank you so much! I really enjoy using this tool and wish to be able to set such priors. I deeply appreciate your help!

Yun
Intelligent Systems Program
University of Pittsburgh
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Re: Setting Beta(Dirichlet) prior when using jsmile EM

Post by mark »

Re. your first point. I implied a uniform distribution (since that's the default in SMILE) but that's indeed not true in general.

Re. your second point. When ESS is set to 200 and the prior mean is 0.75 then alpha=200*0.75 and beta=200*0.25.

Re. your third point. If I remember correctly, that is not supported (you can fix parameters but that is not good enough here). It should not be too hard to add, but someone else would have to help out.
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Setting Beta(Dirichlet) prior when using jsmile EM

Post by marek [BayesFusion] »

I'm afraid the only parameter that you can specify in learning parameters is confidence, which is the equivalent sample size (ESS). This ESS will be for all parameters in the entire network, so it is not possible to specify different ESS for different parameters. I guess you would want at most one ESS per distribution, i.e., one column in the CPT. The reason is that when you learn, you learn a distribution at a time. In any case, SMILE does not allow for setting individual values for ESS. If this is really important for you, you could try to perform multiple learning sessions, learning one node at a time with different ESS by means of Fixed nodes functionality (described in GeNIe manual). This would allow you for different ESSs per node/parents interaction, which might make sense.
I hope this helps.

Marek
Yun
Posts: 20
Joined: Mon Oct 17, 2016 8:34 pm

Re: Setting Beta(Dirichlet) prior when using jsmile EM

Post by Yun »

Dear Marek and Mark,

Thank you all so much for your quick reply! It is very helpful!

-- Following your suggestions, I want to confirm when writing jsmile whether it is correct to do as follows: if I have strong belief that the probability of a random variable X=T is around 0.75 (which I call it prior mean), then I initialize the variable X by "net.setNodeDefinition(X, new double[]{0.75, 0.25}", and then use "em.setEqSampleSize(200)", so that I could get a Beta prior for this random variable (and all variables) with alpha=200*0.75 and beta=200*0.25 (alpha and beta's definition follows wiki's definition for Beta distribution: https://en.wikipedia.org/wiki/Beta_distribution)?

-- For being able to set ESS differently for different random variables, I will try what your suggested! It will be great in the future it can be an option in jsmile?

Thank you so much!
Yun
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Setting Beta(Dirichlet) prior when using jsmile EM

Post by marek [BayesFusion] »

You have it right, I believe.

As far as jSMILE goes, it is open source and you can easily add a wrapper around the SMILE function that you want exposes to Java.
Cheers,

Marek
Post Reply