Calculating values in Python and returning them to pySMILE

[david.s]

Hello,

I am trying to conduct a risk assessment using GeNIe and pySMILE. To do this I set up probabilities for environmental influences, which works great. The result of this is a function node with a discretized temperature distribution. But to proceed with the risk assessment I need to use advanced calculation methods to determine the stress on my system induced by the temperature.

My idea was to read out the data using pySMILE, to perform the calculation needed in Python and then give back the results. So I basically want to use the Equation node but with a costum Python integrated calculation.

Reading out the values of the "Temperature" node works great using:

Code: Select all

net = pysmile.Network()
net.read_file("System.xdsl")
net.update_beliefs()
T_list = net.get_node_value("Temperature")    

Then some calculation is carried out inside python using the function calculate_stress():

Code: Select all

Stress_list = calculate_stress(T_list)

Is there a way to get this Stress_list as result into an equation node?

Kind regards,
David

[shooltz[BayesFusion]]

You can set node equation (as a string) with Network.set_node_equation. The actual formula depends on the type of the output from calculate_stress.

To experiment with various node distributions using SMILE notation, you can use GeNIe's probability visualizer (available from the Tools menu), or go to https://prob.bayesfusion.com, which provides an online version of the visualizer. The back-end of the online visualizer runs on SMILE too.

[shooltz[BayesFusion]]

One more thing: if your calculate_stress function returns a list of values sampled from unknown distribution, then you can try to fit your samples to a metalog distribution. GeNIe/SMILE support Metalog since GeNIe version 4.0. To fit the data, use GeNIe's Tools|Metalog Builder window, or visit an online builder at https://metalog.bayesfusion.com.

[david.s]

Thanks for the reply. The issue is that the integrated functions of GeNIe do not allow the kind of calculation I need. A calculations with multiple matrix multiplications is done in calculate_stress(). As I want to iterate various input values with python and observe the failure probability as outcome this should work automatically, so I am not sure the metalog would be an option. Also there are some very low probability events that I guess would be removed with the metalog distribution.

[shooltz[BayesFusion]]

Is stress_list conditional on any of the nodes in your SMILE model?

[david.s]

Yes, it is conditional on several nodes. For one the "Temperature" node, which is defined as T_list in python, but also another node "Strength" which is conditional to "Temperature" aswell.

[shooltz[BayesFusion]]

What values are is your stress_list? Depending on its structure, the form of the SMILE equation would be different. For example, if the list can be approximated as histogram, you could use the Steps function.

Can you post actual values from stress_list? If it's a very long list, please limit it to first 100 elements.

[david.s]

The first 100 values of stress_list would look e.g. like this:

2.085013436218727989e+01
1.899219629674703569e+01
1.653029092248572240e+01
5.059338097558088165e+00
1.444097813066429659e+01
2.129150571826859206e+01
1.880705604811141285e+01
1.928504591673573287e+01
2.275014283504456003e+01
1.845069407764172453e+01
1.629766262479017414e+01
3.992685116100559650e+00
1.996029785567948167e+01
1.806482786232226800e+01
1.081721502497035914e+02
1.924668676173544313e+01
1.748962463347488949e+01
1.824529904488748855e+01
2.097797621494200726e+01
2.203888168671449677e+01
1.918969329977519678e+01
1.884380087207625465e+01
3.597618084502448355e+00
1.719583593958747869e+01
1.866245264817170479e+01
1.720960593152239682e+01
3.271840654274764347e+03
1.768673460283617516e+01
1.763978408539355769e+01
1.514996001220519339e+01
1.614344887097252013e+01
1.462864809879081562e+01
1.836805303375570020e+01
1.675834622431462861e+01
1.922329050401118877e+01
1.944489550142219869e+01
1.782371668307731838e+01
1.620878951281873981e+01
2.092478443904037988e+01
3.413566623579991756e+03
5.962801884793692807e+03
1.916498231319090095e+01
2.190713363917420153e+01
2.123396950712613673e+01
1.820523282975422674e+01
1.848448038995445586e+01
1.764986809256912011e+03
3.123999001458808834e+03
1.799069221438055877e+01
2.022107365164981019e+01
1.719043952850730506e+01
1.607063796598430372e+01
1.684850428424893209e+01
2.062734946026525620e+02
3.042692518131022705e+00
1.851300021833909781e+01
1.944145147483343550e+01
1.698911529993241132e+01
1.815855588424676981e+01
1.639750356882171189e+01
1.678019964331524250e+01
1.650503312324354255e+01
2.390137425024130735e+01
2.056869941498255372e+01
1.916637796983869890e+01
1.915527371990874173e+01
1.931615428487685904e+01
2.410709550633812796e+03
2.037696649865231535e+01
1.645202619356612672e+01
1.568364275057026092e+01
1.938170079872501006e+01
1.947588946469592841e+01
2.011327208056570015e+01
1.986494674566536389e+01
3.649538117151287064e+00
1.752993976238163398e+01
1.969548844590189418e+01
1.980282594911599148e+01
1.841142903774382589e+01
2.605905189051693469e+03
1.869420151508526828e+01
1.974976250697785929e+01
2.032806277812291285e+01
2.506035375018301238e+03
2.041421519802524642e+01
2.089464614521204112e+03
1.696365472526914076e+01
2.132088459169768058e+01
2.128237865279952246e+01
1.829294991139995830e+01
2.067758175135790211e+01
2.075805015514807295e+01
1.991680379745351814e+01
3.292804437294661657e+00
4.521431166378580201e+03
1.862155613363857753e+01
2.011673094890343449e+01
2.093795635694798207e+01
2.158458002809700105e+01

Some of the values are two magnitudes higher than most. They come from rare events in the order of magnitude of 10E-9. To get this list I increased their probabilities, because otherwise there would not be any outlier in the list, which are important. The Temperature consists of multiple normal distributions for temperatures (T_fire, T_scrape, T_malf, T_operational), which depend on the probability of ocurrence of the event: Temperature = If(Fail_fire,T_fire,If(Fail_scrape,T_scrape,If(Fail_malf,T_malf,T_operational))). The stress_list correlates to the temperature, but not linearly or in any other predictable way. There are parameters I want to assess that change the relationship between T_list and stress_list.

[shooltz[BayesFusion]]

Just to make sure we're on the same wavelength - what of the two options below describes your goal:

a) you want to use stress_list in the same 'system.xdsl' model loaded into your Python program running PySMILE

or

b) you want to derive some SMILE-compatible expression to use in an equation node in other SMILE model, and proceed from that model onwards.

[david.s]

At the moment I am trying to implement option a). Ideally I would be able to use stress_list as values in a node in the same 'system.xdsl' in the same way as if it would have been calculated by an equation node.

I really appreciate your effort to settle my enquiry.

[shooltz[BayesFusion]]

Unfortunately, currently we don't have an option to defined an 'external function' in SMILE. By external function I mean SMILE calling into user-specified code, which processes inputs (in practice, a current sample value from parent node(s)), and returns a value which is treated by SMILE in the same way as an output from its built-in functions available in equations (Normal, Uniform, Sin, etc).

SMILE has 'custom functions', which can be composed from existing built-ins, but this does not solve the performance problem you've mentioned earlier.

I've added an entry in our bug/feature tracker with the idea of 'external functions'. However, we can't give you any estimate on when this functionality would be available. An inquiry from business customers would certainly move it up in the queue :)