Equation Learning

The front end.
Post Reply
martinfriebe
Posts: 4
Joined: Wed May 31, 2017 7:50 pm

Equation Learning

Post by martinfriebe »

Hello there,
I'm working on a big engineering PhD project and would like to discuss some direction I can take with my project. Ultimate goal is to find the underlying 'meta'-equations of engineering systems.

My Project: Being further into an engineering design phase and having a lot design information available, it is easy to do any sort of performance analysis. My approach is to do an exhaustive modelling of a particular system (let's say a car) in all its possible design configurations. Next, I extract those variables and learn them through a BN learning algorithm. (I have completed this so far and done a small decision making analysis and I'm currently trying to write it up)

Problem: my BN, even though it learns...it just learns the already known, it does not generate any new knowledge

Question: Is it possible to let a BN sort of extrapolate learned knowledge? - My finding from my exhaustive case study could be for instance that for a linear increase in 'car travel range' I need a linear 'increase in fuel tank size'. Currently I can only see that for the exhaustively modeled scenarios, BUT if I could transform the CPT into an actual function...then I could explore many more scenarios outside outside my already explored design space. Is there a way to get those equations out of GeNie?

A throught: My (unsatisfying) provisional workaround would be to use the Naive Bayes algorithm and to pour the distributions into excel and compute the trend-lines. From the trend-lines I could get the equations, use the computed equations in equation nodes and expand my BN model. Would be cool though if GeNie has an answer as this is a multi-dimensional problem ..

Would be great to get someones else's thought on this!
Cheers,
Martin
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Equation Learning

Post by marek [BayesFusion] »

Martin,

A great direction for work, not necessarily specific to Bayesian networks but rather to all system modeling tools. We are modeling precisely to be able to understand the system. Bayesian networks are closely related to systems of simultaneous equations. We are about to release a version of GeNIe that will allow for modeling hybrid systems, so you will be able to specify interactions among variables by both equations and CPTs. Our long-term vision is to be able to learn equations as well, not only CPTs, as we have (actually, the PC algorithm when applied to multivariate normal distributions learns linear equations capturing interactions among variables). At the moment, however, you will have to learn equations outside of GeNIe.

As far as deriving interactions between variables that are located far from each other in the model, please have a look at the following paper that I co-authored with my former doctoral student, Tsai-Ching Lu:

http://www.pitt.edu/~druzdzel/psfiles/ejor09.pdf

Figure 10 illustrates the idea that you are talking about. We have it in GeNIe, along with ImaGeNIe, although it is hidden deeply (just not reliable enough to be put to industrial use).

Do keep us updated about your work!

Marek
martinfriebe
Posts: 4
Joined: Wed May 31, 2017 7:50 pm

Re: Equation Learning

Post by martinfriebe »

Thanks Marek,

there is so much interesting stuff, I wish my day would have more than 24 hours. Well, currently I have completed the case study modelling, I got all data and modeled a BN. The network seems plausible and there is probably already enough material to write up a decision making process on that developed model, but we are pushing for the underlying equations right now.

After doing some reading I came up with following approach: First, I have let GeNie build a Network, which already passed significance checks, but also took some 'background knowledge'. Second step is checking my data columns for significance (again) and correlation in Excel. Third step is to run the data (correlating nodes in GeNie) through a linear regression analysis in Excel, retrieving the functional relationship so they can be fed back into GeNie as functions.

Trying to get it done tomorrow or latest the day after
martinfriebe
Posts: 4
Joined: Wed May 31, 2017 7:50 pm

Re: Equation Learning

Post by martinfriebe »

Hello Marek,

I've been working on my network and trying to replace all (20!?) nodes through linear equations, since GeNie does currently not support hybrid models yet. However, this renders my model very simplistic. When do you think you will release the version supporting hybrid networks?

Cheers,
Martin
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Equation Learning

Post by marek [BayesFusion] »

Martin,

The planned release date is late June/early July. I will be happy to send you a link to our beta version. Will follow this up with a personal note.
Cheers,

Marek
martinfriebe
Posts: 4
Joined: Wed May 31, 2017 7:50 pm

Re: Equation Learning

Post by martinfriebe »

A quick question regarding the data learning constraints: I have a bunch of runs with close to 1500 variables, is there a limitation of how many variables I can use? I have the feeling that I came across the information once, but I can't really recall where it was. We are able to cut down the number of variables manually, but I'd like to know the maximum number of variables possible, so I can create the right output format/input to GeNIe.
marek [BayesFusion]
Site Admin
Posts: 430
Joined: Tue Dec 11, 2007 4:24 pm

Re: Equation Learning

Post by marek [BayesFusion] »

There is no theoretical limitation and the algorithms should run with 1,500 variable. Bayesian search is pretty much looking for a needle in a haystack, so please do use restarts. PC will hit the combinatoric complexity of the independence tests, so it may take a long time. My advice is to set a very low significance level and also a time limit so that the algorithms come back. Do let us know how it is going and please send us the most challenging data sets so that we can improve the software.
Cheers,

Marek
Post Reply