does PySMILE has the capability to use multiple CPU?

The engine.
Post Reply
Yang Yajie
Posts: 34
Joined: Thu Mar 19, 2020 11:49 am

does PySMILE has the capability to use multiple CPU?

Post by Yang Yajie »

Dear sir or madam,

I will run my code on the HPC in our school, I would like to ask that Does PySMILE has the capability to use multiple CPU? Does SMILE allow to use multiple CPU's?

kind regards!
Yajie
piotr [BayesFusion]
Site Admin
Posts: 60
Joined: Mon Nov 06, 2017 6:41 pm

Re: does PySMILE has the capability to use multiple CPU?

Post by piotr [BayesFusion] »

You can use PySMILE code with Python multiprocessing package, like below:

Code: Select all

import pysmile
import pysmile_license
import multiprocessing
import time

ds = pysmile.learning.DataSet()
ds.read_file("[path_to_dataset_file]")

def learn_net_from_dataset(ds):
    bs = pysmile.learning.BayesianSearch()
    bs.learn(ds)
    

if __name__ == '__main__':
    starttime = time.time()
    processes = []
    
    for i in range(1000):
        p = multiprocessing.Process(target=learn_net_from_dataset, args=(ds,))
        processes.append(p)
        p.start()
    
    for process in processes:
        process.join()
        
    print('Learning with multiprocessing took {} seconds'.format(time.time() - starttime))
    
    starttime = time.time()
    
    for i in range(1000):
        learn_net_from_dataset(ds)
        
    print('Learning without multiprocessing took {} seconds'.format(time.time() - starttime))
    
I hope it helps :)
Yang Yajie
Posts: 34
Joined: Thu Mar 19, 2020 11:49 am

Re: does PySMILE has the capability to use multiple CPU?

Post by Yang Yajie »

thank you. but your answer is related to structure learning. I create the network by myself and I need to learn the parameter of this created network in HPC.
Is it possible to use more than one CPU cores when making the parameter learning?
And Is it possible to use multiple cores in the single CPU when making parameter learning?
shooltz[BayesFusion]
Site Admin
Posts: 1417
Joined: Mon Nov 26, 2007 5:51 pm

Re: does PySMILE has the capability to use multiple CPU?

Post by shooltz[BayesFusion] »

Is it possible to use more than one CPU cores when making the parameter learning?
And Is it possible to use multiple cores in the single CPU when making parameter learning?
The answer to both question is negative. SMILE currently does not use multiple threads in its inference and learning algorithms.

We had at one point a proof of concept code running in the Hadoop cluster performing EM learning. This required some changes in the core library and the support from SMILE's Java wrapper.
Post Reply