Continuous variables

The front end.
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Continuous variables

Post by Christian »

Hello,

I found in the tutorial an explaination about "Continuous variables". Is it possible to use these in GeNIe? As I found no way to set a chance - node to a "Continuous" one. I only can set true or false.

Maybe if that is possible I could also set something like 40% true, 60% false as Evidence. But I think I need to build that using an helping additional chance node. I am right?

Thank you,

Christian
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: Continuous variables

Post by shooltz[BayesFusion] »

We have the implementation for continous (equation-based) variables. It's still in the early stages and there's no direct way of creating them in GeNIe (there's a magic key combination, though :) )
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

As I need the continuous variables, is it the same to just define one node (the one that I consider to be the continuos Variable) and set the percent value directly?

For example: I want a continuous variable with 60%. Do I have to set value (state0) to 60% and "antivalue"(state1) to 40%?

If I want to change the continuous variable to 20%, do I have to set the Value (state0) to 20% and the "antivalue" (state1) to 80%?

Would that be the same as a continuous variable if this node is not dependet on other nodes? Or is this something that should not be done?

I just don't want to descretize the value that needed to be set into so many categories. So is this a workaround?

If not, could you please give me the "magic key" for the continuous variables? :D

Thank you,

Christian
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

Christian wrote:Maybe if that is possible I could also set something like 40% true, 60% false as Evidence. But I think I need to build that using an helping additional chance node. I am right?
This is called soft-evidence, but it is not supported in GeNIe (or SMILE). It is possible to implement by using an additional chance node, as you described.
Christian wrote:For example: I want a continuous variable with 60%. Do I have to set value (state0) to 60% and "antivalue"(state1) to 40%?
This is very different from having a continuous variable in the network. Soft-evidence means that you define a prior probability distribution on the states of a (non-root) node instead of using probabilities from the cpt. The node still has only two states, state0 or state1, and child nodes can only be conditioned on those states. In case of continuous variables the value of a node could be 60 (or 40), and a child could be conditioned on that. In your proposed approach, this is impossible.
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

Okay, I never knew that this is called soft evidence. Is that the same as "virtual evidence" in netica?

Can I do the following as workaround?


Node1:
True 60%
False 40%

Node 2 has True activated.

Then I build 6 datasets out of that one with Node1 true, Node2 true AND 4 with Node1 False, Node2 true.

Then I will have the relationship. Okay, I will need to add 10 datasets for everyone.

Or do you know an algorithm that I can use to calculate the needed values on my own, using soft evidence?

Because I have about 2 million datasets to be learned and to get the correct relationship I need to create about 1 million datasets for everyone of these 2 Million datasets. Smile is a bit slow with this amount of datasets :)

Christian

P.S.: Sorry for the late answer. I forgot to check "Notify me when a reply is posted"
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

Sorry, but I don't understand what you are trying to do. Can you please elaborate?
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

I am trying to simulate a soft evidence.

For example if I have a node with 60% state0 and 40% state1 I build 10 examples, 6 with state0 to 100% and 4 with state1 to 100%

After training this net I should have the 60%/40% for that node.

Now let asume I have two nodes that are connected (Node 1 is parent of node 2):

Node1:
state0 (N1S0)
state1 (N1S1)

Node2:
state0 (N2S0)
state1 (N2S1)

Now I want to build
N1S0 = 60%, N1S1 = 40 If N2S0 = 90% / N2S1 = 10%

Now I create 100 examples:
54: N1S0 = true, N1S1 = false, N2S0 = true, N2S1 = false
6: N1S0 = true, N1S1 = false, N2S0 = false, N2S1 = true
36: N1S0 = false, N1S1 = true, N2S0 = true, N2S1 = false
4: N1S0 = false, N1S1 = true, N2So = false, N2S1 = true


Is that now simulated soft evidence?
So this way is a work around to learn soft evidence with smile?

Now it is important that I build 100 examples for every dataset. Otherwise the relationship between the datasets would be gone.

My problem is now to build the correct datasets if I want to simulate soft evidence training with much more nodes and states than the two above.

So my question is/was if you know an easier way to archieve that. Or if you know a good algorithm.

Thank you,

Christian
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

I think soft evidence is related to inference and not learning. I constructed an example with two nodes where initially none of the nodes have evidence (see net1.jpg). Suppose soft evidence becomes available about node 2, namely P(node2=state0)=0.2 and P(node2=state1)=0.8. This can be implemented by attaching an extra node3 to node2 (see net2.jpg). To get the right soft evidence probabilities in node2, we use Bayes' rule to calculate the probabilities in the cpt for node3. I also attached the xdsl network if you want to have a look at the cpts. I'm not sure if my approach is fool proof, but in this simple case it seems to work fine.
Attachments
net.xdsl
(2.38 KiB) Downloaded 775 times
net2.jpg
net2.jpg (10.94 KiB) Viewed 20750 times
net1.jpg
net1.jpg (7.26 KiB) Viewed 20750 times
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

ahh, okay - I understand. Thank you very much for your detailed examples. Thats a good approach. Then that's the way to go for me.

But if I need soft evidence in a parent node I can just change the weights?
For example in your net1.jpg - Node1, if I want 60% state0 and 40% state1 I can just set it, right?

So it is only called soft evidence if a child node is meant?
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

Correct, when it is a root node you can just change the prior probabilities of that node.
joergweber79
Posts: 21
Joined: Thu Jan 15, 2009 4:22 pm

Post by joergweber79 »

mark wrote:I think soft evidence is related to inference and not learning. I constructed an example with two nodes where initially none of the nodes have evidence (see net1.jpg). Suppose soft evidence becomes available about node 2, namely P(node2=state0)=0.2 and P(node2=state1)=0.8. This can be implemented by attaching an extra node3 to node2 (see net2.jpg). To get the right soft evidence probabilities in node2, we use Bayes' rule to calculate the probabilities in the cpt for node3. I also attached the xdsl network if you want to have a look at the cpts. I'm not sure if my approach is fool proof, but in this simple case it seems to work fine.
Hi Mark, can you please provide a formula, which shows how the cpt of node3 can be calculated? Using the bayes rule we get:

P(Node3=State0|Node2=State0)=P(Node2=0|Node3=0)*P(Node3=0)/P(Node2=0)

which is as value: 0.2*0.39/0.65=0.118 <- the value of the probability table in node3

The value of P(Node3=0) is the marginal probability and has the value 0.39.

Normally the marginal probability is computed as:

P(Node3=0)=P(Node3=0|Node2=0)*P(Node2=0) + P(Node3=0|Node2=1)*P(Node2=1)

But we don't have probabilities of Node3 at this point. How can I create a formula which gives me back the 0.118 for the cpt of node3 in dependency of P(Node2=0), P(Node2=1) and desired "softevidence"-value(80, 20) ???

Can you please help me?
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

Probably the easiest way to calculate the probabilities is not to use Bayes' rule, but to do the following. Because of the soft evidence, P(node2=state0|node3=state0)=0.2=P(node2=state0,node3=state0)/P(node3=state0). Both terms can be calculated by marginalizing the joint distribution. Let P(node3=state0|node2=state0)=x and P(node3=state0|node2=state1)=1-x. After writing everything out (which is tedious) you can solve for x and find the probabilities in the CPT. Let me know if you need more details.
joergweber79
Posts: 21
Joined: Thu Jan 15, 2009 4:22 pm

Thanks for advice

Post by joergweber79 »

Hello Mark,

thank you for your advice. Using the formulas
P(node3=state0|node2=state0)=x and P(node3=state0|node2=state1)=1-x

...i have derived a formula for the marginal probability of node3=state0:

P(Node3=state0)=P(Node2=state1)*P(Node2=state0)/(P(Node2=state1|Node3=state0)*P(Node2=state0)+P(Node2=state0|node3=state0)*P(node2=state1))
=0.3855

...using the bayes rule we get

P(Node3=state0|Node2=state0)=P(Node2=state0|Node3=state0)*P(Node3=state0)/P(Node2=state0)
=0.1186

Is that formula mathematically correct? In comparison to the example you provided, i have a small difference in the results. Using the formula above genie exactly displays the values (0.2/0.8 ) in case of soft evidence.

But I have one more question:

Are there any inconsistencies, if i would use this "workaround" for an intermediate node? (Not an leaf node)

Thanks in advance!

Cheers,

Joerg
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

To calculate the marginal probability of Node3, normally you would marginalize the joint distribution, i.e., P(Node3) = \sum_{Node1, Node2} P(Node1, Node2, Node3). My previous post described how to obtain the joint distribution. I don't know how you derived your formula, but it seems to calculate the marginal probability. I think this approach should also work for intermediate nodes, although the formulas will become more complicated.
ben_mrad
Posts: 10
Joined: Sat Mar 27, 2010 2:53 pm

Soft Evidence and Fuzzy Bayesian Network

Post by ben_mrad »

Hello,

I am student on second year Master's of search "New Technology and Dedicated Computer System" and I work on Fuzzy Bayesian Network (FBN) ...
I think that this axis is very similar to the notion " soft evidence "..
What you think of that?
And if you can explain to me the functioning of " Soft Evidence "?

Thank you in advance
Post Reply