Horizons in Probabilistic Programming and Bayesian Analysis

Representations:

• Hierarchical models
• Hidden Markov models
• Graphical models
• Non-parametric Bayes (distributions over functions)

Inference Approaches:

• Brute-force calculation
• Random-walk Monte Carlo sampling: Metropolis, Gibbs, ABC
• Gradient-based simulation: HMC, NUTS
• Optimization
• Variational approaches

Python Software:

• PyMC: Metropolis and adaptive Metropolis samplers
• PyMC3: next-generation sampling and fitting algorithms for Bayesian models
• emcee: MCMC
• bnpy: Bayesian non-parametric ML
• Bayes Blocks: variational Bayes
• BayesPy: general posterior inference
• libpgm: Bayesian probability graphs
• Pythonic Bayesian Belief Network Framework: eBay's belief network framework
• PyStan: HMC, VB
• pgmpy: probabilistic graphical models

Questions:

• What are people currently working on?
• Which exciting new methods should we be implementing?
• Which technologies are going to be able to help make probabilistic programming in Python easier and more effective?
  • PyCUDA
  • Theano
  • OpenCL

Ideas:

• Gallery of models for PyMC 3
• Adding parallel tempering to PyMC 3

Linked list:

• FlyMC
• Factorie
• Queso
• Kombine
• Parallel tempering
• Bridge estimation
• Crosscat
• BayesDB
• PyMC-Spark
• abpmc
• Black box variational inference
• Expectation propagation as a way of life