Confidence in model

The engine.
Post Reply
romeo
Posts: 7
Joined: Mon Dec 10, 2007 5:27 pm

Confidence in model

Post by romeo »

(This question was asked/answered before on the old forum and pity it is not accessible. Apologies for not storing the response)


How do I estimate the confidence in a dataset on which my model is trained? I believe the answer was that using EM I can assign a confidence level to datasets but that is not quite the same thing as confidence in the model.
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Re: Confidence in model

Post by mark »

romeo wrote:(This question was asked/answered before on the old forum and pity it is not accessible. Apologies for not storing the response)


How do I estimate the confidence in a dataset on which my model is trained? I believe the answer was that using EM I can assign a confidence level to datasets but that is not quite the same thing as confidence in the model.
With EM you can assign a confidence level to the network that will be updated using the data. A confidence level of 100 means that the current network is based on 100 records. Does this answer your question?
romeo
Posts: 7
Joined: Mon Dec 10, 2007 5:27 pm

Post by romeo »

Hi Mark

I was asking about estimating the confidence in the training dataset (*not* the EM confidence level), there was a response on the old forum about something to do with cross-validation.

Could you elaborate?
mark
Posts: 179
Joined: Tue Nov 27, 2007 4:02 pm

Post by mark »

romeo wrote:Hi Mark

I was asking about estimating the confidence in the training dataset (*not* the EM confidence level), there was a response on the old forum about something to do with cross-validation.

Could you elaborate?
The confidence in the training data is usually not in question, unless you obtained the data through noisy sensors. And even then you could use a model to account for the noise. I guess you're trying to ask how to assess the quality of the parameters (starting with random parameters) after learning them in the network using EM. For this you could use, for example, cross-validation or bootstrapping.
Post Reply