Genie has problems with big nets

The front end.
Post Reply
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Genie has problems with big nets

Post by Christian »

Hello,

if I build a network with more than average 25 million values I can calculate, edit all the values and so on. But as soon as I want to save this net, GeNIe is crashing.

So it seems that smile is ok with that big networks, but GeNIe not?
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: Genie has problems with big nets

Post by shooltz[BayesFusion] »

Can you describe the configuration of the network, so I can try to reproduce the problem?
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

Hello,

sorry for my late answer, I was out of town lately. I have build an example for you.

To reproduce the error, do the following:

1) Open the network
2) Connect Node6 to Node 1.
3) Press F5 to calculate. This should work fine every time.
4) Press Save As and save the network.

Most of the time GeNIe crashes now. If it does not crash I am unable to load the network from that file.

If you need a much more sure crashing situation, just enter some more states in Node6 between 1) and 2).

Using these large networks, GeNIe can caluclate them, but not save/load them. My guess is, that the error is a result as soon as GeNIe uses more than 700MB to save. But for calculation GeNIe does not use that amount of memory as it seems.
Attachments
crashing_network.zip
(152.42 KiB) Downloaded 896 times
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Post by shooltz[BayesFusion] »

Thanks for the bug report. I was able to reproduce the problem on my machine. The problem is caused by very large size of CPT in Node1. The XDSL writer converts each node probability table to string representation before writing it to disk. In this case, there's not enough memory used for actual CPT and the temporary string. The solution of course is to write directly to the file.

Unfortunately, I can't give you the estimate for the bugfix now; this is not a trivial change, especially on the parser side. Stay tuned.
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Post by shooltz[BayesFusion] »

I have modified the XDSL writer and parser to handle your network. The changes were extensive, so I need more time to test it thoroughly. New binaries should be available in the first week of 2008.
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Post by shooltz[BayesFusion] »

shooltz wrote:New binaries should be available in the first week of 2008.
Let's call it first _full_ week of 2008. But they're coming :)
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

:D

It's okay, it is done when it is done :D
I can wait. So take the time you need. Hey, it is a free software, so everything I get and every response of you and Marc is a bounty.
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Post by shooltz[BayesFusion] »

I have working/tested GeNIe and SMILE now. However, since the changes in the XDSL parser were extensive, I plan to test it for one or two more days.

Send me a private message if you'd like to download binaries before they go final/public.
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Post by shooltz[BayesFusion] »

GeNIe 2.0.2932.0 is ready for download.
Christian
Posts: 44
Joined: Wed Nov 28, 2007 12:32 pm

Post by Christian »

thank you very much, I will give it a try.
Post Reply