[Basic] - How to use beta density functions in equation nodes correctly

The front end.
Post Reply
mlfee
Posts: 5
Joined: Wed Mar 29, 2017 10:28 am

[Basic] - How to use beta density functions in equation nodes correctly

Post by mlfee »

Dear all,

I am a beginner user of GeNIe and would like to apply Beta Density Functions correctly in to Genie.

I am trying to represent the following scenario where I have 3 jars of coins. One jar is called X1 and the scond jar is called X2|X1 = 1 and X2|X1=2.

if the coin from jar X1 is flipped and shows a head, it is represented as X1 = 1
if the coin from jar X1 is flipped and shows a tail, it is represented as X1 = 2

hence the jar "X2|X1 = 1" is used when the coin from jar X1 is flipped and shows a head.
and the jar "X2|X1 = 1" is used when the coin from jar X1 is flipped and shows a tail.

The experiment is run 7 times so the following results is obtained:

Case X1 X2
1 1 1
2 1 1
3 1 1
4 1 2
5 2 1
6 2 1
7 2 2

and wish the express the experiment in the following bayesian network (photo attached), and GeNIe filed attached.

I start with assumption that all beta density functions have equal frequency of Beta ( 1,1)

Based in the results above, I can identify that for the node F11 ( of the state of X1), my node should be defined as beta(a= heads, b= tails) => Beta(1+4=5,1+3=4).

Also, for the node F21(which is the number of times X2 = 1 or X2 = 2 given that X1 = 1) => Beta (4,2)

Also, for the node F22 (which is the number of times X2 = 1 or X2 = 2 given that X1 = 2) => Beta (3,2).

If I just draw the nodes as per the attached figure, I get the Node X2 being defined as Node X2 = F21 + F22 + X1 as it seems to be an automated additive function when links are drawn...and end up with a mean 'probability' or >1 which is not correct I know but I can't figure out how to obtain a probability value for a BBN with Beta Density Functions.

I know that with the 'a' and 'b' values, I can work out the probablity of X1, and X2|X1 by hand but I would like to update the network from observing evidence of each run case and changing the a and b values and getting an updated probaility.

Thank you so much!
Attachments
BetaDistribution_v0.xdsl
(1.96 KiB) Downloaded 411 times
BBN-Learning.JPG
BBN-Learning.JPG (30.36 KiB) Viewed 5877 times
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: [Basic] - How to use beta density functions in equation nodes correctly

Post by marek [BayesFusion] »

I will be glad to try to help you. However, I would like to ask you first what the variables X1 and X2 represent. Clearly, variable F11 represents the probability of obtaining a head in a single coin toss. Variables F21 and F22 represent the probability of obtaining a head in a single toss of a coin drawn from jars "X2|X1 = 1" and "X2|X1=2" respectively. If you tell me what you intend X1 and X2 to express, I will be glad to help.

I got a little confused by the sentence "I get the Node X2 being defined as Node X2 = F21 + F22 + X1 as it seems to be an automated additive function when links are drawn..." A sum of the parents is indeed the default function placed in the definition of a node. This is similar to placing uniform distribution in a new discrete node. We do this because GeNIe's approach has been that the model under construction is always syntactically correct, so we place a default definition there to satisfy syntactic correctness but you should replace this definition with whatever you intend to do. It seems to me that you should use some kind of conditional functions in your calculation, e.g., If() function. However, I will be glad to help you created the correct definitions once I know what you want the nodes X1 and X2 to express.
Cheers,

Marek
mlfee
Posts: 5
Joined: Wed Mar 29, 2017 10:28 am

Re: [Basic] - How to use beta density functions in equation nodes correctly

Post by mlfee »

Dear Marek,

Firstly, thank you so much for responding and offering your assistance!
f you tell me what you intend X1 and X2 to express, I will be glad to help
Having re-visit the diagram from Neapolitan's book(I had obtained the diagram from this book), I think that the purpose of X1 is to highlight it as an event with data input from the variable F11. Perhaps there is no way to show in a textbook in an animated way of what data a node consists of ( It's not like when the user can double click on a node and see what data it consists of :-)). I think the author wants to show more distinctly, graphically, that the event, or node, X1 has a probability distribution of F11. Likewise, for the event or node X2, it has the probability distributions of F21 and F22 and is dependent on the event X1.

Strictly speaking, the node X1 would take on the distribution of F11, so I would say that F11=beta(5,4) after the 7 runs of coin toss. Initial F11 = beta(1,1)

As I understand from Neapolitan's textbook, a beta distribution of a data set can be represented as (or attached as an image, to be able to see the equation with more ease)

P(d) = (Γ(N) / (Γ(N + M)) * (Γ(a + s)Γ(b + t)) / (Γ(a)Γ(b)) , where Γ(x) = (x-1)!

where d is the data set ( of 7 throws of coins), and has a beta distribution with parameters a, b, N = a + b, s, t, M= s+t. P(d) is the relative frequency with which we will obtain data d when we repeatedly sample coins with replacement from the urns and toss each sample pair 7 times.

What I would like is for X1 = p(d) = (Γ(2) / (Γ(2 + 7)) * (Γ(1 +4)Γ(1 + 2)) / (Γ(1)Γ(1)) with the beta density function values of F11 ( Initial a=1, b=1, N=2, s=4, t=2, M=7) ). I notice on GeNIe that there is only the Gamma Log function.

(I tried using R to compute and by typing x1<- beta(5,4) and I entered 'x1' in order to have R return the value of x1, R gives to be 0.003571, which is computed by the gamma function equation as above.)

Instead of typing the individual a,b,N,s,t,M values in to the P(d) equation at each time I update the prior, is there a way i can have X1 be updated by changing the values at F11=Beta (a,b)?

As for X2, I would like to compute the probability of the data by multiplying the probability of the data on X1, the probability of the data on X2 when X1 = 1, and the probability of the data on X2 when X1 = 2, so that would be

X2 = (P(d) from F11) * ((P(d) from F21) * (P(d) from F22)

=1.488e-5

F21 = beta(4,2). ( Initial F21 = beta(1,1) )
F22= beta(3,2) (Initial F22 = beta (1,1)

(For reference)
P(d) from F21 = (Γ(2) / (Γ(2 + 4)) * (Γ(1 + 3)Γ(1 + 1)) / (Γ(1)Γ(1))
P(d) from F22 = (Γ(2) / (Γ(2 + 3)) * (Γ(1 + 2)Γ(1+ 1)) / (Γ(1)Γ(1))

Conditional Probability tables

Another question I have is that how can I see the condition probability table at X2. I have worked out by hand based on the beta function values, by taking the ratio a/(a+b) and obtain:

P(X1 = 1, X2 = 1) = P(X2 = 1|X1 = 1)P(X1 = 1) = (2/3)*(5/9) = 10/27
P(X1 = 1, X2 = 2) = P(X2 = 2|X1 = 1)P(X1 = 1) = (1/3)*(5/9) = 5/27
P(X1 = 2, X2 = 1) = P(X2 = 1|X1 = 2)P(X1 = 2) = (3/5) *(4/9) = 4/15
P(X1 = 2, X2 = 2) = P(X2 = 2|X1 = 2)P(X1 = 2) = (2/5)(4/9) = 8/45


I am now clear on this part
This is similar to placing uniform distribution in a new discrete node. We do this because GeNIe's approach has been that the model under construction is always syntactically correct, so we place a default definition there to satisfy syntactic correctness but you should replace this definition with whatever you intend to do.
Thank you very much for your time and I will respond if my post above is unclear. Thank you for your time spent on helping me (and everyone else) with our problems. I truly appreciate your help
Attachments
eqn-gammafunction.JPG
eqn-gammafunction.JPG (50.2 KiB) Viewed 5857 times
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: [Basic] - How to use beta density functions in equation nodes correctly

Post by marek [BayesFusion] »

Thank you for the additional explanation and the reference to Neapolitan's book. I believe I understand your problem better now. What looks like a BN in Neapolitan's book is not really a BN. He wants to show how the parameters of X1 and X2 are calculated based on the Beta distributions (gray nodes). In other words, the darkened circles that look like variables/nodes (F11, F21 and F22) are shown as Beta distributions that influence the parameters and not the nodes X1 and X2. Hence, the figure does not really show a Bayesian network and Bayesian network algorithms will be pretty much useless there.

I have recreated your model in such a way that it gives the meaning to X1 and X2 as the events Heads and Tails (1 and 2 respectively) in a single trial. The marginals show the probability of these events happening. I reproduced the process of generating the samples, assuming your Beta distributions. Please note that the marginals in the model (distribution over 1s and 2s) correspond very nicely to the marginals in your data set -- after all the parameters come from Beta distributions, which were obtained from the data set.

I have also created a discrete model that contains the parameters calculated by Neapolitan. Please note that the two models show the same marginals, which demonstrates that the procedure is correct.

I'm attaching both models. We are about to release GeNIe 2.2., which will have the hybrid BN capability, i.e., discrete and continuous networks in the same model. In fact, I created the two models in one, using the prototype that is undergoing extensive pre-release tests. Please stay tuned.

Please let me know if you have any additional questions.

Marek
Attachments
Beta Discr.xdsl
(1.17 KiB) Downloaded 403 times
Beta Cont.xdsl
(1.98 KiB) Downloaded 466 times
mlfee
Posts: 5
Joined: Wed Mar 29, 2017 10:28 am

Re: [Basic] - How to use beta density functions in equation nodes correctly

Post by mlfee »

Dear Marek,

Thank you for your time and effort in creating the models, and for a clear explanation of what the grey/white circles mean. Neapolitan has captioned the figure I have used in the first post as a "Binomial augmented Bayesian Network"..so your comments are right in that they are not really a Bayesian Network.
I have recreated your model in such a way that it gives the meaning to X1 and X2 as the events Heads and Tails (1 and 2 respectively) in a single trial.
Thank you for your models, they help to clarify a lot. I do wonder about the equation at X1, which is X1 = 1 + Bernoulli (1-F11). I understand the use of 1 is to be able represent the states as "1" and "2" ( defined by me/Neapolitan/user) by shifting (+1) from the default of "0" and "1". However, for the Bernoulli(p) function, I would like to know why p is represented as '1-F11". I noted that F11 = Beta ( 5,4) and represented as 0.55 ( which is the probability of Heads). By following the workflow, I would imagine that the Bernoulli (p) would refer to probability of Heads and the complement '1-p' would refer to probability of Tails. I also tried to write X1= 1+ Bernoulli (F11) while maintaining F11 as Beta (5,4) and naturally got the probability values flipped. Perhaps my reasoning is incorrect...?

Another question I have is that the Beta distribution is actually a special case of the Dirichlet Distribution. Assuming I would like to consider three states (or maybe more, in other scenarios) of the coin toss, such as State 1 = Heads, State 2 = Tails, State 3 = Coin landing on its edge so that its neither heads or tail. Is there an equivalent equation of Dir(3,5,1) like Beta (5,4) I could use in GeNIe? Perhaps then extending the if/else equations could be tricky as it is no longer a 2-state analysis where Bernoulli could be used.

Or is it easier to compute the CPTs from the Dir (3,4,1) data and set up the model as in the Discrete model example you have attached.

Thank you for your time and I am looking forward to the release of Genie 2.2!
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: [Basic] - How to use beta density functions in equation nodes correctly

Post by marek [BayesFusion] »

You are right about the X1 = 1 + Bernoulli (1-F11): I have added 1 in order to shift the result from the interval [0;1] to the interval [1;2]. The 1-F11 and 1-F12 are because Beta distributions in the model are for the probability of heads and you have defined heads as 1 and tails as 2. This is flipped, so I had to reverse the resulting probability. Please note that Beta produces a number in the interval [0;1]. Does this make sense?

There is no Dirichlet distribution in the function set. Thank you for suggesting it -- we will add it in the next release of the program. In the meantime, perhaps you can write it as an equation involving other distributions. Please note that GeNIe gives you a complete modeling freedom -- there is no limitation on what equations you use in the definition.
I hope this helps,

Marek
Post Reply