Network size limitation of SMILE package

[BayesFusionUser123]

Dear Community,

I am following the PDF version of the tutorial for SMILE wrappers, specifically for python language.


I am trying SMILE package on our real-world dataset.

Our dataset has 1654 nodes, 2965 edges.
Among the 1654 nodes, there are 605 continuous variables and 1049 categorical variables.

To make learning parameter EM algorithm work on this network, I discretize all continuous variables into 5 bins.

After I learn the parameter of this network from a dataset. I try to print out the posterior of a target variable.

I tried the following code:

Code: Select all

hybrid_net.update_beliefs()
print_posteriors(hybrid_net, 0)

It gives error message like below:

Code: Select all

--------------------------------------------------
SMILEException Traceback (most recent call last) Cell In[84], line 1 
----> 1 hybrid_net.update_beliefs() 2 print_posteriors(hybrid_net, 0) 

SMILEException: SMILE Error -42 in function Network.UpdateBeliefs

I checked the error code and found:
https://support.bayesfusion.com/docs/SM ... codes.html

The -42 means `DSL_OUT_OF_MEMORY`


My question:
Does it means SMILE cannot deal with network of this size?
What is the largest size of network can SMILE deal with?
How can I solve this issue?

Thank you very much.

[shooltz[BayesFusion]]

Error -42 is caused by insufficient memory in exact inference algorithm. This algorithm creates temporary data structure (the join tree), which can become very large, depending on the connections in the network.

You can switch to approximate sampling inference (EPIS), or set explicit targets in the network if you're only interested in the subset of node probabilities.

[BayesFusionUser123]

Hi,

"set explicit targets in the network if you're only interested in the subset of node probabilities"

print_posteriors() function is like below:

Code: Select all

def print_posteriors(net, node_handle):
    node_id = net.get_node_id(node_handle)
    if net.is_evidence(node_handle):
        print(f"{node_id} has evidence set ({net.get_evidence_id(node_handle)})")
    else :
        posteriors = net.get_node_value(node_handle)
        for i in range(0, len(posteriors)):
            print(f"P({node_id}={net.get_outcome_id(node_handle, i)})={posteriors[i]}") 

When I run print_posteriors(net, 0), it is only for target variable with node_handle 0

I want to clarify that it looks like it is not that I did not set explicit targets that cause memory issue.
It is the update_beliefs() function call that reports out of memory error.

Below is my test on a small network:

Code: Select all

manipulate_net = pysmile.Network()
# load the network with learned parameters.
manipulate_net.read_file("./simulated_data/simulated_data_em_cont.xdsl")

# If I call print_posteriors only for target 2, it will reports error

Code: Select all

print_posteriors(manipulate_net, 2)

Code: Select all

---------------------------------------------------------------------------
SMILEException                            Traceback (most recent call last)
Cell In[33], line 1
----> 1 print_posteriors(manipulate_net, 2)

Cell In[22], line 6, in print_posteriors(net, node_handle)
      4     print(f"{node_id} has evidence set ({net.get_evidence_id(node_handle)})")
      5 else :
----> 6     posteriors = net.get_node_value(node_handle)
      7     for i in range(0, len(posteriors)):
      8         print(f"P({node_id}={net.get_outcome_id(node_handle, i)})={posteriors[i]}")

SMILEException: SMILE Error 1 in function Network.GetValue

I must firstly update_beliefs() and then print_posteriors only for target variable.

Code: Select all

manipulate_net.update_beliefs()
print_posteriors(manipulate_net, 2)

Then it gives correct result:

Code: Select all

P(T=T___0)=0.54084
P(T=T___1)=0.45916

Below is the situation on a real world network

If I run

Code: Select all

hybrid_net_with_para.update_beliefs()
print_posteriors(hybrid_net_with_para, 1)

The hybrid_net.update_beliefs() that cause it to have -42 error.

Code: Select all

SMILEException                            Traceback (most recent call last)
Cell In[144], line 1
----> 1 hybrid_net.update_beliefs()
      2 posteriors = hybrid_net.get_node_value(1)

SMILEException: SMILE Error -42 in function Network.UpdateBeliefs

(2)
(2.1)
Could you please point me to some tutorial with sample code or document to do "approximate sampling inference (EPIS)"?
(2.2)
I found that SMILE wrapper document lacks lots of information. Did I missed anything, any other document resource?

I have read the EPIS related material in the GeNIe document. But I cannot find any documentation in the wrapper document about how to set EPIS.

In the Network methods list I found two methods get_bayesian_algorithm() and set_bayesian_algorithm()
I use get_bayesian_algorithm() to get current algorithm, it prints <BayesianAlgorithmType.LAURITZEN: 0>.
Could you please show me how to set it to EPIS?

Thank you very much.