Discrepancy between simulated CDF and discretized probabilities

The front end.
Post Reply
nzsk
Posts: 4
Joined: Thu Aug 29, 2019 8:27 am

Discrepancy between simulated CDF and discretized probabilities

Post by nzsk » Thu Feb 06, 2020 3:44 pm

I'll try to express as clearly as possible my question, which seems rather complex to me. I have a hybrid network with >30 nodes simulating phosphorus concentrations in a river catchment. The simulated values are broadly sensible, with mean 0.009, min=0.00002 and maximum=0.12. I'm trying to simulate the probability of concentrations falling within regulatory limits, especially those below 0.07. According to the CDF, over 99% of simulations fall into this bracket. However, when I trigger the discretisation by instantiating one of the internal nodes in the network, the marginal probability of values falling below 0.07 is extremely low (1%), while the marginal probability of higher values is 99% - this is very counter-intuitive to me.

There appears to be a discrepancy in values that can be seen in the Histogram and the CDF under the 'Values' tab and values shown in the 'Elicitate' window under the 'Discretization' tab. The elicitate window shows some extemely high values, whilts these are not apparent among the simulations. Could this be a cause of the problem and if so, how does this work?

I have some screenshots that I could provide, if this helps to answer the query. Thank you very much for your help.

marek [BayesFusion]
Site Admin
Posts: 273
Joined: Tue Dec 11, 2007 4:24 pm

Re: Discrepancy between simulated CDF and discretized probabilities

Post by marek [BayesFusion] » Thu Feb 06, 2020 5:58 pm

I will be happy to try to help. I may need to play with it myself. Would you be willing to post the piece of the network in question or send it to me (Email will work best: [ADMIN: removed email to protect the innocent - please use private forum message instead]). You can obfuscate your network if you are worried about your IP. It would be most helpful to me if you could reduce the network to the bare minimum where this is happening.
Cheers,

Marek

nzsk
Posts: 4
Joined: Thu Aug 29, 2019 8:27 am

Re: Discrepancy between simulated CDF and discretized probabilities

Post by nzsk » Wed Feb 12, 2020 1:40 pm

Dear Marek, many thanks for your reply. I have sent you the network via a private forum message as suggested. I have sent you the full network as I can't see how it could be simplified whilst keeping the network properties. However, the problem should be clear to see in the final network node. Maybe we can discuss together how to simplify it, if needed. Looking forward to your reply. With best wishes Miriam

marek [BayesFusion]
Site Admin
Posts: 273
Joined: Tue Dec 11, 2007 4:24 pm

Re: Discrepancy between simulated CDF and discretized probabilities

Post by marek [BayesFusion] » Thu Feb 13, 2020 7:11 pm

Got the network, thanks! I'm not sure which variable you are referring to as your outcome variable. Is it "Pconcentration mg L-1"? I tried to discretize it into more intervals to check where the samples are falling and that gave me some insight -- many samples are not really far from the bottom values. Please try that and try to discretize the variables that feed into it into more intervals. I suspect that the main culprit are problems related to rough discretization, although I can't say for sure, of course, without knowing the domain better.

If playing with the discretizations does not help, perhaps getting together online will work, as you will be able to guide me through the network and the physical quantities involved. One problem is that I am about to go on a trip to Asia and will be EST+11. If it can wait, it will be wonderful. Please contact me through EMail. The forum software cuts out EMail addresses but you can find me at marek at bayesfusion.com.
Cheers,

Marek

Post Reply