What is the difference between causal models and directed graphical models?

Question

What is the difference between causal models and directed graphical models? What is the difference between causal relationships and directed probabilistic relationships? More concretely, what would you put in the interface of a DirectedProbabilisticModel class, and what in a CausalModel class? Would one inherit from the other?

Tying

I thought that if machine-learning was a tag then machine learning questions were fair game: surely this is computer science related. I'm trying to make sense of this: ftp.cs.ucla.edu/pub/stat_ser/r350.pdf – Tying

Answer 1

There are two types of causal model: interventional models and counterfactual models. All directed graphical models can reason observationally. An interventional model is a directed graphical model that can reason with observational and interventional evidence. A counterfactual model can reason with observational, interventional, and counterfactual evidence (interventions whose source is inferences within the model).

In a private email a couple years ago, Pearl wrote me that:

By definition, a model is a list of assumptions, and assumptions are never "known to be true". They may be substantiated by theory, or data, or experiments. But their position in the hierarchy is determined by what they claim, not by where they came from.

Tying

Answer 2

Causality by Judea Pearl is the book to read.

The difference is that one is causal and the other is merely statistical. Before dismissing me as a member of the tautology club, hear me through.

A directed probabilistic relationship (AKA a complete set of Conditional Probability Tables , AKA Bayesian Network) only contains statistical information. Meaning that anything you can infer from the Joint Probability table you can infer from the directed probabilistic relationship, nothing more, nothing less. The two are equivalent.

A causal relationship is something else entirely. A causal relationship (AKA Causal Bayesian Network) must specify what happens under any variable intervention. Intervention is when a variable is forced to a value outside of the normal influences of the model. This is equivalent to replacing the conditional probability for the forced variable (or variables, but we consider just one for simplicity) with a new table in which the variable takes its forced value with probability one.

If this does not make sense, please follow up and I will clarify.

This section added to address Neil's questions in the comments

Neil asks:

How can you determine the direction of directed probabilistic relationships without performing interventions? In other words, doesn't the directed graphical model have causal information in it (i.e., information about probabilities conditional on interventions?)

You can determine the direction of directed probabilistic relationships by making additional non-statistical assumptions. These assumptions commonly include: assuming no hidden variables, and the really important one, assuming that the conditional independence relationships found in the joint distribution are stable (meaning they exist not by chance or cancellation). Bayesian Networks do not make these assumptions.

For details of how to recover the directions research the IC, PC, and IC* algorithms. I believe the specific details of IC are covered in: "A Theory of Inferred Causation"

Synopsize

How can you determine the direction of directed probabilistic relationships without performing interventions? – Tying

In other words, doesn't the directed graphical model have causal information in it (i.e., information about probabilities conditional on interventions?) – Tying

@Neil, I added a new section to my answer to address your questions. – Synopsize

Thanks for the link. I read the paper, and I think I can rephrase my question as follows: What would you put in the interface of a DirectedProbabilisticModel class, and what in a CausalModel class? Would one inherit from the other? – Tying

@Neil, the CausalModel class inherits from the DirectedProbabilisticModel class. The DPM class has interfaces that let you specify what P(x|y) you want. The CM class has interfaces that let you specify what P(x|y, do(z)) you want, where "do(z)" stands for some intervention. As there may be zero interventions the addIntervention interface would only be for CM, and the other interfaces would be common. – Synopsize

My understanding that both are both directed GMs and causal models are basically the same. The difference is that in a causal model, the arrows must go in the correct causal way, while in a directed GM, the arrows must only respect dependencies. This means that every causal model is also a directed GM. But you say a causal model is something else entirely. – Sunbathe

@Sunbathe I'm talking specifically about Causal Bayesian Networks as defined by Judea Pearl in the book Causality. Other people may mean something different when they say "causal model". Just like saying "Graphical Model" doesn't necessarily mean you are talking about a Bayesian Network. – Synopsize

Your answer is still perfectly correct and helped me come to my current understanding. I hope you don't mind that I've since thrown up a simple answer that summarizes its main points since you answered me a decade ago. – Tying

Regarding your first sentence, "Causality, by Judea Pearl, is the book to read": Pearl has since written a new book, "The Book of Why", 2018, in which he confesses and corrects errors he made in earlier years. On page 331, he writes, "I even said so in the first edition (2000) of my book Causality. This was one of the three greatest blunders of my career." The Book of Why is fascinating. – Court

Answer 3

If I understand this post correctly, casual models and directed graphical models (Bayesian networks) aim at different stages of the workflow. A casual model is a way of assigning dependencies such that they reflect causation. Bayesian networks provide us with inference techniques. So, one can perform estimation using something different. On the other hand, one can model Bayesian networks using different techniques than SCM.

If you dig into it deeper, please let us know, because I don't fully understand the subject of SCM (while I'd like to :).

Dorking

I really think that the only difference between causal models and directed graphical models is that causal models remain informative after interventions. In other words, they implement the interface I've defined above. – Tying

Answer 4

directed graphical models are a way of encoding causal relationships between variables. probabilistic graphical models are a way of encoding causality in a probabilistic manner. I would recommend reading this book written by Judea Pearl who is one of the pioneers in the field (whom I see you refer to in the paper you mentioned in the comment).

a directed graph is simply a graph (nodes and edges) which is directed (edges have directions). causal models are models which tell you how variables affect each other, one way of doing that is using directed graphs. AI research has shown that deterministic causal relationships are not sufficient to encode knowledge of the world around us because it is too messy. That is why probability was added to the picture.

Detraction