Greetings everyone, I am new in ML and modelling, I would like to ask whether if I can use the confusion matrix of multi-class variable (>2 classes) to compute the precision and sensitivity for the variable as the GeNIe provides the accuracy values only.
Regards the ROC, can the GeNIe plot such type of graph? can you provide the steps if so?
Thank you.
Confusion Matrix and ROC
-
- Site Admin
- Posts: 432
- Joined: Tue Dec 11, 2007 4:24 pm
Re: Confusion Matrix and ROC
Sensitivity, specificity, ROC and calibration are meaningful for binary classes only and this is what GeNIe does normally -- you can select the class that you are interested in in a dialog of the verification tabs. What GeNIe does in case of multi-class variables is calculation of sensitivity and specificity in such a way that all remaining classes are lumped together. The confusion matrix shows you everything, so you can calculate the sensitivities and specificities of each of the class yourself. ROC and calibration is also for binary classes. Here also, all other classes are lumped together.
I hope this helps.
Cheers,
Marek
I hope this helps.
Cheers,
Marek
Re: Confusion Matrix and ROC
Alright Marik.. thanks for the explanation.. at the moment I face a problem which is the Bayesian search algorithm (learn network) cannot find connections (nodes) among the 11 varaibles. Though I used the same file and procedures I used before last week. Do you encounter such a situation?.. I am new in using the software ,perhaps I have a problem in my settings which I am not aware of or did not notice though I used the same file and set of data last week, and I was able to get the network via mentioned algorithm but as I tried today I couldn't get the connections..
thanks
thanks
-
- Site Admin
- Posts: 432
- Joined: Tue Dec 11, 2007 4:24 pm
Re: Confusion Matrix and ROC
Hard to say what is going on. If a learning algorithm does not create arcs, then the simplest explanation is that the variables in the data set are independent. Please check this using any statistical package. If the data are continuous, you can look at the scatterplot. While any software can contain a bug, GeNIe has been beaten quite heavily for over 20 years and we have had learning built in for 15 years now. I am using it regularly myself. If you cannot figure out what is happening (please try yourself first!), please send me your data and I can have a look at them.
Cheers,
Marek
Cheers,
Marek
Re: Confusion Matrix and ROC
Hello Marek,,, I reinstall the software again and it works again. I belief I had the earlier version of the software as I installed it in 2014 which did not have the ROC. thanks for the advice concerning the bug issue. problem solved
For the validation of network there are three options including (1) test, (2) leave one out and (3) cross validation. If I understand correctly the cross validation and the leave one out options will divide validation data to learning and testing, and then do the checking. Thereof, for the test option will be defied as to use the whole validation set and cross check it with the learned model based on my training set. Does this the right explanation for the test option?
thanks
For the validation of network there are three options including (1) test, (2) leave one out and (3) cross validation. If I understand correctly the cross validation and the leave one out options will divide validation data to learning and testing, and then do the checking. Thereof, for the test option will be defied as to use the whole validation set and cross check it with the learned model based on my training set. Does this the right explanation for the test option?
thanks
-
- Site Admin
- Posts: 432
- Joined: Tue Dec 11, 2007 4:24 pm
Re: Confusion Matrix and ROC
This is correct -- Marek