Hi everybody,
i have created my BN. it supposed to be a coutinious Target. But Genie doesn't support this case. Anyway I want to validate my BN. There are 3 Options.
Q1: Can I create 2 Datasets. the first one for paramter learning and the second one just for TEST ONLY? Does that way make a sence??
Q2: Which methode is for my case to use? Cross-Validation or leave one out? What does each one of them exactly do?
Thanks alot
Validation
-
- Site Admin
- Posts: 430
- Joined: Tue Dec 11, 2007 4:24 pm
Re: Validation
Hi Ramzi,
The best way to use validation for continuous variables will be to discretize the target variable, I guess. Let me try to answer your two questions.
Q1: Leaving aside a subset of records for testing is a perfectly fine procedure, known as hold-out method. The main problem with it is that it leaves out records that could be used for learning for testing. k-fold cross-validation is more efficient in that respect.
Q2: There is a simple description of the three methods in GeNIe manual. Leave-one-out is an extreme case of k-fold cross-validation, when k=n, where n is the number of records. It gives more accurate results at a higher computational expense. If you can afford it time-wise, I recommends leave-one-out over k-fold cross-validation.
I hope this helps,
Marek
The best way to use validation for continuous variables will be to discretize the target variable, I guess. Let me try to answer your two questions.
Q1: Leaving aside a subset of records for testing is a perfectly fine procedure, known as hold-out method. The main problem with it is that it leaves out records that could be used for learning for testing. k-fold cross-validation is more efficient in that respect.
Q2: There is a simple description of the three methods in GeNIe manual. Leave-one-out is an extreme case of k-fold cross-validation, when k=n, where n is the number of records. It gives more accurate results at a higher computational expense. If you can afford it time-wise, I recommends leave-one-out over k-fold cross-validation.
I hope this helps,
Marek