Validation

The front end.
Post Reply
ramzi9572
Posts: 6
Joined: Mon Apr 26, 2021 10:24 am

Validation

Post by ramzi9572 »

Hi everybody,

i have created my BN. it supposed to be a coutinious Target. But Genie doesn't support this case. Anyway I want to validate my BN. There are 3 Options.

Q1: Can I create 2 Datasets. the first one for paramter learning and the second one just for TEST ONLY? Does that way make a sence??

Q2: Which methode is for my case to use? Cross-Validation or leave one out? What does each one of them exactly do?

Thanks alot
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Validation

Post by marek [BayesFusion] »

Hi Ramzi,

The best way to use validation for continuous variables will be to discretize the target variable, I guess. Let me try to answer your two questions.

Q1: Leaving aside a subset of records for testing is a perfectly fine procedure, known as hold-out method. The main problem with it is that it leaves out records that could be used for learning for testing. k-fold cross-validation is more efficient in that respect.

Q2: There is a simple description of the three methods in GeNIe manual. Leave-one-out is an extreme case of k-fold cross-validation, when k=n, where n is the number of records. It gives more accurate results at a higher computational expense. If you can afford it time-wise, I recommends leave-one-out over k-fold cross-validation.

I hope this helps,

Marek
Post Reply