em.learn() Execution time in Jsmile

[ohernan]

I have a BN with 12 input nodes and 1 output node. The em.learn() algorithm is taking 30 minutes. Is this execution time normal?

[shooltz[BayesFusion]]

The execution time depends on the structure of your networks (large CPTs will slow down the learning), the file size (obviously the greater the row count, the longer the EM runtime) and the amount of missing data.

Can you compare the jSMILE and GeNIe runtimes for your network and datafile?

[ohernan]

Hi,
Thanks for answering,
I have a network with 17 input nodes (each one with 3 states) and 1 output node with 2 states (The CPT have more that 40,000,000 of columns :-() neither, jsmile or GENIE can calculate the probabilities, GENIE throw an "out of memory error".
Thanks


Oscar Hernán Franco Bedoya
Assistant Researcher
Department of Service and Information System Engineering (ESSI)
Universitat Politècnica de Catalunya (UPC)
Barcelona, Spain

[shooltz[BayesFusion]]

The CPT have more that 40,000,000 of column

That's definitely a workload for 64-bit SMILE on a machine with tons of RAM.

[marek [BayesFusion]]

Brute force will not do here and your problem will be faced by any Bayesian network software. SMILE is one of the most efficient and fastest packages there are. Please note the the size of CPT grows exponentially with the number of parents and your CPT is indeed quite challenging :-). Try some techniques for structural changes to your network, such as "parent divorcing". They can reduce the size of your model drastically.

Marek

[ohernan]

Thanks a lot for your recommendation, I am going to try this approach.
Regards
Oscar