does PySMILE has the capability to use multiple CPU?

[Yang Yajie]

Dear sir or madam,

I will run my code on the HPC in our school, I would like to ask that Does PySMILE has the capability to use multiple CPU? Does SMILE allow to use multiple CPU's?

kind regards!
Yajie

[piotr [BayesFusion]]

You can use PySMILE code with Python multiprocessing package, like below:

Code: Select all

import pysmile
import pysmile_license
import multiprocessing
import time

ds = pysmile.learning.DataSet()
ds.read_file("[path_to_dataset_file]")

def learn_net_from_dataset(ds):
    bs = pysmile.learning.BayesianSearch()
    bs.learn(ds)
    

if __name__ == '__main__':
    starttime = time.time()
    processes = []
    
    for i in range(1000):
        p = multiprocessing.Process(target=learn_net_from_dataset, args=(ds,))
        processes.append(p)
        p.start()
    
    for process in processes:
        process.join()
        
    print('Learning with multiprocessing took {} seconds'.format(time.time() - starttime))
    
    starttime = time.time()
    
    for i in range(1000):
        learn_net_from_dataset(ds)
        
    print('Learning without multiprocessing took {} seconds'.format(time.time() - starttime))
    

I hope it helps :)

[Yang Yajie]

thank you. but your answer is related to structure learning. I create the network by myself and I need to learn the parameter of this created network in HPC.
Is it possible to use more than one CPU cores when making the parameter learning?
And Is it possible to use multiple cores in the single CPU when making parameter learning?

[shooltz[BayesFusion]]

Is it possible to use more than one CPU cores when making the parameter learning?
And Is it possible to use multiple cores in the single CPU when making parameter learning?

The answer to both question is negative. SMILE currently does not use multiple threads in its inference and learning algorithms.

We had at one point a proof of concept code running in the Hadoop cluster performing EM learning. This required some changes in the core library and the support from SMILE's Java wrapper.