DBN validation accuracy

The front end.
Post Reply
safaa
Posts: 31
Joined: Sat May 24, 2014 7:00 am

DBN validation accuracy

Post by safaa »

Hello !
I have
1. made DBN with 3 time slice and then train dataset=11261 case (the training used the default settings of genie)

2. I have performed the following experiments (the target node called POS)
(temporal arc only on the node pos
dbn.rar
(16.58 KiB) Downloaded 340 times
)
a. validate (2kfold)with class node pos at time t1 gives =79% accuracy
b.validate (2kfold)with class node pos at time t2 gives =81% accuracy
(
dbnc.rar
(189.79 KiB) Downloaded 324 times
)
a. validate (2kfold)with class node pos at time t gives =76% accuracy
b.validate (2kfold)with class node pos at time t1 gives =84% accuracy
c.validate (2kfold)with class node pos at time t2 gives =84% accuracy

now im working on making the time slice =10 and see if the validation accuracy will be increased.
my questions:
>>I know that there exist so much factors affect the accuracy. and of course the dataset is one of them. in the second experiment I have count the frequency of all time slices for the pos node in the training set. and I found that pos at time t has the largest frequency although it has the smallest accuracy,why?

>>is my approach right? what should i do to have better performance?
>>I understood validation , but can we say that if validation get better accuracy then inference we have better accuracy?
>>when we make parameter learning should we set the target node?
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: DBN validation accuracy

Post by shooltz[BayesFusion] »

in the second experiment I have count the frequency of all time slices for the pos node in the training set. and I found that pos at time t has the largest frequency although it has the smallest accuracy,why?
I'm sorry, but I can't understand this question. Please rephrase and/or post the training set.
safaa
Posts: 31
Joined: Sat May 24, 2014 7:00 am

Re: DBN validation accuracy

Post by safaa »

>>why would the accuracy for a node at time t0 be lower than at time t=1?
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: DBN validation accuracy

Post by marek [BayesFusion] »

I am somewhat confused by your experiment. I assume your data set is a time series since you have learned from it a DBN. In a time series, it is not surprising that you can determine the class better as time progresses. If it is a static data set, then I suspect you are dealing with over-fitting the data. These are my first impressions about what you are doing.
Cheers,

Marek
safaa
Posts: 31
Joined: Sat May 24, 2014 7:00 am

Re: DBN validation accuracy

Post by safaa »

Hello Marek!
this is sample from my data set
dbn3ttest13_9.txt
(99.12 KiB) Downloaded 420 times
.
I don't understand
marek wrote:over-fitting the data
.
I would be grateful if you tell me what do you think? the validation accuracy are not clear for me. if you would like i can send you the details. and other experiments
Post Reply