Pyro: Deep Universal Probabilistic Programming
Authors: Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman
JMLR 2019 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To demonstrate that Pyro meets our design goals, we implemented several state-of-the-art models... We used the MNIST data set... To demonstrate that Pyro s abstractions do not reduce its scalability by introducing too much overhead, we compared our VAE implementation with an idiomatic Py Torch implementation... Figure 3 shows that the relative performance gap between the Pyro and Py Torch versions is moderate... |
| Researcher Affiliation | Collaboration | Eli Bingham EMAIL... Uber AI Labs Uber Technologies, Inc. 555 Market St., San Francisco, CA, 94103... Noah D. Goodman EMAIL Uber AI Labs 555 Market St., San Francisco, CA 94103 Stanford University, Stanford, CA, USA |
| Pseudocode | Yes | Figure 1: A complete Pyro example: the generative model (model), approximate posterior (guide), constraint speciļ¬cation (conditioned_model), and stochastic variational inference (svi, loss) in a variational autoencoder. encoder is a torch.nn.Module object. def model(): loc, scale = torch.zeros(20), torch.ones(20) z = pyro.sample("z", Normal(loc, scale)) w, b = pyro.param("weight"), pyro.param("bias") ps = torch.sigmoid(torch.mm(z, w) + b) return pyro.sample("x", Bernoulli(ps)) def guide(x): pyro.module("encoder", nn_encoder) loc, scale = nn_encoder(x) return pyro.sample("z", Normal(loc, scale)) def conditioned_model(x): return pyro.condition(model, data={"x": x})() optimizer = pyro.optim.Adam({"lr": 0.001}) loss = pyro.infer.Trace_ELBO() svi = pyro.infer.SVI(model=conditioned_model, guide=guide, optim=optimizer, loss=loss) losses = [] for batch in batches: losses.append(svi.step(batch)) |
| Open Source Code | Yes | Pyro s source code is freely available under an MIT license and developed by the authors and a community of open-source contributors at https://github.com/uber/pyro and documentation, examples, and a discussion forum are hosted online at https://pyro.ai. |
| Open Datasets | Yes | We used the MNIST data set and 2-hidden-layer MLP encoder and decoder networks with varying hidden layer size #h and latent code size #z for the VAE and the same data set of digitized music4 to train the DMM. (Footnote 4) This is the JSB chorales data set from http://www-etud.iro.umontreal.ca/~Eboulanni/icml2012 |
| Dataset Splits | No | We used the MNIST data set and 2-hidden-layer MLP encoder and decoder networks with varying hidden layer size #h and latent code size #z for the VAE and the same data set of digitized music4 to train the DMM. To demonstrate that Pyro s abstractions do not reduce its scalability by introducing too much overhead, we compared our VAE implementation with an idiomatic Py Torch implementation.5 After verifying that they converge to the same test ELBO, we compared the wall-clock time taken to compute one gradient update, averaged over 10 epochs of GPU-accelerated mini-batch stochastic gradient variational inference (batch size 128) on a single NVIDIA GTX 1080Ti. |
| Hardware Specification | Yes | We compared the wall-clock time taken to compute one gradient update, averaged over 10 epochs of GPU-accelerated mini-batch stochastic gradient variational inference (batch size 128) on a single NVIDIA GTX 1080Ti. |
| Software Dependencies | No | Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research... Pyro uses stochastic variational inference algorithms and probability distributions built on top of Py Torch, a modern GPU-accelerated deep learning framework. |
| Experiment Setup | Yes | optimizer = pyro.optim.Adam({"lr": 0.001})... averaged over 10 epochs of GPU-accelerated mini-batch stochastic gradient variational inference (batch size 128) |