Pyro vs Pymc? What are the difference between these Probabilistic Programming frameworks?

Question

I used 'Anglican' which is based on Clojure, and I think that is not good for me. Bad documents and a too small community to find help. Also, I still can't get familiar with the Scheme-based languages. So I want to change the language to something based on Python.

Maybe Pyro or PyMC could be the case, but I totally have no idea about both of those.

• What are the difference between the two frameworks?
• Can they be used for the same problems?
• Are there examples, where one shines in comparison?

Posthaste

I'd vote to keep open: There is nothing on Pyro [AI] so far on SO. It remains an opinion-based question but difference about Pyro and Pymc would be very valuable to have as an answer. – Artist

In Terms of community and documentation it might help to state that as of today, there are 414 questions on stackoverflow regarding pymc and only 139 for pyro. – Govan

Answer 1

(Updated for 2022)

Pyro is built on PyTorch. It has full MCMC, HMC and NUTS support. It has excellent documentation and few if any drawbacks that I'm aware of.

PyMC was built on Theano which is now a largely dead framework, but has been revived by a project called Aesara. PyMC3 is now simply called PyMC, and it still exists and is actively maintained. Its reliance on an obscure tensor library besides PyTorch/Tensorflow likely make it less appealing for widescale adoption--but as I note below, probabilistic programming is not really a widescale thing so this matters much, much less in the context of this question than it would for a deep learning framework.

There still is something called Tensorflow Probability, with the same great documentation we've all come to expect from Tensorflow (yes that's a joke). My personal opinion as a nerd on the internet is that Tensorflow is a beast of a library that was built predicated on the very Googley assumption that it would be both possible and cost-effective to employ multiple full teams to support this code in production, which isn't realistic for most organizations let alone individual researchers.

That said, they're all pretty much the same thing, so try them all, try whatever the guy next to you uses, or just flip a coin. The best library is generally the one you actually use to make working code, not the one that someone on StackOverflow says is the best. As for which one is more popular, probabilistic programming itself is very specialized so you're not going to find a lot of support with anything.

Lieutenant

I also think this page is still valuable two years later since it was the first google result. Firstly, OpenAI has recently officially adopted PyTorch for all their work, which I think will also push PyRO forward even faster in popular usage. One thing that PyMC3 had and so too will PyMC4 is their super useful forum (discourse.pymc.io) which is very active and responsive. Regard tensorflow probability, it contains all the tools needed to do probabilistic programming, but requires a lot more manual work. – Marital

Good disclaimer about Tensorflow there :). I was furiously typing my disagreement about "nice Tensorflow documention" already but stop. – Anon

I'm currently using BUGS or Stan, and I wish these Python-based PPLs were a bit more intuitive, but I guess it is what it is. Seems like Pyro might have a bit easier learning curvce. – Exotic

Same Evan. PyMc3 looked familiar, and the 'hello world' OLS was straightforward. However, the multiprocessing support seems hopelessly broken. – Ashe

Update as of 12/15/2020, PyMC4 has been discontinued. Instead, the PyMC team has taken over maintaining Theano and will continue to develop PyMC3 on a new tailored Theano build. See here for PyMC roadmap: github.com/pymc-devs/pymc3/wiki/Timeline – Hopi

The latest edit makes it sounds like PYMC in general is dead but that is not the case. As per @Hopi PYMC4 is no longer being pursed but PYMC3 (and a new Theano) are both actively supported and developed. – Guillemette

Seconding @Guillemette , PyMC3 has become PyMC and Theano has a been revived as Aesara by the developers of PyMC. So PyMC is still under active development and it's backend is not "completely dead". As the answer stands, it is misleading – Vercingetorix

updated my answer to reflect what JJR4 and Richard Border mentioned – Lieutenant

"[Pyro] has excellent documentation and few if any drawbacks that I'm aware of." - it has a very steep learning curve and exposes a lot of very technical details to the user, as evidenced by highly experienced probabilistic ML folks being unable to get basic models to run without asking for help on the forum. Technically it is excellent, but the end-user-facing interface, documentation, and tutorials could use a lot of love (or a simpler-to-use wrapper library). Whenever I decide to implement some simple model in pyro, I get stuck for days or weeks on very low-level implementation stuff. – Preteritive

pymc (great usability / documentation / user interface) now supports the excellent and extremely fast numpyro and blackjax backends for MCMC sampling; that is probably the path I would go for a new project nowadays. – Preteritive

Answer 2

From here

Pyro is a deep probabilistic programming language that focuses on variational inference, supports composable inference algorithms. Pyro aims to be more dynamic (by using PyTorch) and universal (allowing recursion).

Pyro embraces deep neural nets and currently focuses on variational inference. Pyro doesn't do Markov chain Monte Carlo (unlike PyMC and Edward) yet.

Pyro is built on pytorch whereas PyMC3 on theano. So you get PyTorch’s dynamic programming and it was recently announced that Theano will not be maintained after an year. However, I found that PyMC has excellent documentation and wonderful resources. Another alternative is Edward built on top of Tensorflow which is more mature and feature rich than pyro atm. Authors of Edward claim it's faster than PyMC3.
I guess the decision boils down to the features, documentation and programming style you are looking for.

Frustum

Does this answer need to be updated now since Pyro now appears to do MCMC sampling? – Radom

@Radom yes, but it should also be emphasized that Pyro is only in beta and its HMC/NUTS support is considered experimental. – Short

PyMC4 will be built on Tensorflow, replacing Theano. – Croquet

Now NumPyro supports a number of inference algorithms, with a particular focus on MCMC algorithms like Hamiltonian Monte Carlo, including an implementation of the No U-Turn Sampler. Additional MCMC algorithms include MixedHMC (which can accommodate discrete latent variables) as well as HMCECS. github.com/pyro-ppl/numpyro – Antidromic