This section reviews two useful structural transformation of Bayesian networks that preserve the join probability distribution represented by the network: arc reversal and node marginalization.
Arc reversal
Formally speaking, the lack of an arc between two nodes x and y that denotes (possibly conditional) independence between x and y. Less formally, but more intuitively, an arc in Bayesian networks denotes direct probabilistic influences. It is a good idea to draw arcs in causal direction and let them denote causal relationships. It may happen in the process of building a Bayesian network that you draw an arc from a node x to a node y and later realize that you would rather have the arc oriented from y to x. Another situation in which you may want to change the direction of an arc is when performing expert elicitation and realizing that while the causal direction is from x to y, P(y|x) is easier to estimate. The operation of arc reversal is a structural transformation of a Bayesian network that allows for changing the direction of an arc but at the same time preserves the joint probability distribution represented by the network.
Consider the simple Bayesian network used in the Hello GeNIe! example
To change the direction of the arc, right-click on it and choose Reverse.
The result is a Bayesian network that represent the same joint probability distribution between the two variables but has the arc pointing in the opposite direction.
Compare the CPT of the node Success of the venture before (left) and after (right) the operation of arc reversal.
Similarly, different is the CPT of the node Expert forecast before (left) and after (right) the operation of arc reversal.
The CPTs are recalculated in such a way that they yield the same joint probability distribution over all nodes in the network.
There is an additional complication related to reversing an arc: Because the structure of the network encodes explicitly independences in the domain, reversing an arc violates the pattern of independences. To account for this, in general the two nodes at the opposite ends of the arc need to inherit their parents. In a network with many nodes and connections among them, the operation of arc reversal may, thus, lead to a significantly more complex structure. Consider the following Bayesian network:
Reversing the arc between the nodes Ratio of Debts to Income and Credit Worthiness transforms the graph to the following form
It happens so that the operation of arc reversal loses some structural information about the network and, in particular, about the independences that the structure of the network represents. Because of this, reversing the same arc back does not necessarily restore the original graph
Our advice is to use this operation rarely, if possible. It preserves the joint probability distribution represented by the model but it may lead to loss of a very important property, notably the structure of the underlying graph.
Node marginalization
It may happen that we want to simplify a model by means of removing a variable from it. If we want to preserve the joint probability distribution over the nodes in the network, we can use the operation of marginalization. Deleting the variable instead of marginalizing it in would lead to loss of the numerical properties of the network.
Consider again the simple Bayesian network used in the Hello GeNIe! example
To marginalize the node Success of the venture, right-click on it and choose Marginalize.
GeNIe issues a warning that the marginalized node will be deleted from the network. Pressing No gives us a chance to retract from the operation.
Pressing Yes, however, removes the node Success of the venture while modifying the CPTs of the remaining nodes so that they preserve their ability to represent the joint probability distribution. The CPT of the node Expert forecast is transformed to the following
which is what would be its marginal distribution with the node Success of the venture present.
Sometimes, when the marginalized node is involved in complex interactions with other nodes, the operation of marginalization may introduce additional arcs, which have as a goal preservation of the dependencies that are lost by removing the node. Consider the following Bayesian network:
Marginalizing the Worth transforms the graph to the following form
Please note the new arcs between Income and Future Income and Assets and Future Income.
Marginalization of a node may in addition lead to change of the type of its children. When a child of the marginalized node is Deterministic or NoisyAdder, it will be converted to a chance general node. The same may happen with a child node that is a NoisyMax - whenever a new arc is added to that node, it is no longer a Noisy-Max node in general.
The operation of marginalization is powerful but it leads to loss of information, so we advise caution in using it.