Validation + Verification BN

The front end.
Post Reply
sykes
Posts: 10
Joined: Tue Jan 15, 2008 8:03 pm

Validation + Verification BN

Post by sykes »

hi,

I've constructed a BN using GeNIe and I'm looking for an "accepted" approach on validating the network that I have designed.

Would you happen to know of a tool (or technique or method) that explains the way in which one validates a BN?

Thanks in advance,
Ed Sykes
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Post by marek [BayesFusion] »

You need to tell us more :-). Have you built the network based on data or on expert knowledge? If on data, you can use part of the data to learn the network and a part to validate it -- this is called cross-validation. If it is based purely on expert knowledge, there are many techniques that you can use, for example validate it in practice, test it with your expert based on some typical scenarios, etc. Lots of possibilities, not systematic enough to list here :-). Cheers,

Marek
sykes
Posts: 10
Joined: Tue Jan 15, 2008 8:03 pm

Post by sykes »

Dear Marek,

Thanks so much for your help. I'm working on constructing a BN (influence diagram) to determine when to interrupt a user as s/he is doing a computer based task. I've attached the BN. It is 100% based on expert knowledge at this time. If you could point me to some resources that validate it (from a scientific perspective). I was thinking of scenario walk-throughs with the Expert etc. but I was thinking there may be more thorough approaches, more "accepted" approaches, etc.

Thanks again for your help.

Cheers,
Ed Sykes
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Post by marek [BayesFusion] »

A Americans say, "The proof is in the pudding" -- the ultimate performance of your system is the best way of verifying it. I think a scenario work-through would be one simple method. Take a few imagined cases of users who should be interrupted (accodring to your expert) and see whether your model recommends interruption. Then take a few cases of users who should not be interrupted and see whether yoru model reacts correctly. Sensitivity analysis might identify those variables and parameters in your model that are critical. Have you looked at the connections strengths (expressed by thickness of arcs) in GeNIe? That should also help. In general, however, the problem is hard, as you are dealing with a model, an artifact, and are asking the question "Is my model good". To make a statement about the model you need to compare it to reality. I hope this helps.
Cheers,

Marek
sykes
Posts: 10
Joined: Tue Jan 15, 2008 8:03 pm

Post by sykes »

Dear Marek,

Thanks again for your help. I think I have a better understanding now what I need to do. The stregth of relationship looks like a very good feature in GeNIe that I will check into.

Cheers,
Ed
Post Reply