A Heavy-Tailed Algebra for Probabilistic Programming
Authors: Feynman T. Liang, Liam Hodgkinson, Michael W. Mahoney
NeurIPS 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results confirm that inference algorithms that leverage our heavy-tailed algebra attain superior performance across a number of density modeling and variational inference (VI) tasks. 4 Experiments We now demonstrate that GGA-based density estimation yields improvements in tail estimation across several metrics. |
| Researcher Affiliation | Academia | Feynman Liang Department of Statistics University of California, Berkeley EMAIL Liam Hodgkinson School of Mathematics and Statistics University of Melbourne, Australia EMAIL Michael W. Mahoney ICSI, LBNL, and Department of Statistics University of California, Berkeley EMAIL |
| Pseudocode | Yes | Algorithm 1: GGA tails static analysis pass Data: Abstract syntax tree for a PPL program Result: GGA parameter estimates for all random variables frontier [rv : Parents(rv) = ]; tails {}; while frontier = do next frontier.pop Left(); tails[next] compute GGA(next.op, next.parent); frontier frontier + next.children(); end return tails |
| Open Source Code | Yes | To illustrate an implementation of GGA for static analysis, we sketch the operation of the PPL compiler at a high-level and defer to the code in Supplementary Materials for details. |
| Open Datasets | Yes | super (superconductor critical temperature prediction dataset [23] with n = 256 and d = 154); who (life expectancy data from the World Health Organisation in the year 2013 [41] with n = 130, d = 18); air (air quality data [14] with n = 6941, d = 11); and blog (blog feedback prediction dataset [7] with n = 1024, d = 280). |
| Dataset Splits | No | All experiments are repeated for 100 trials, trained to convergence using the Adam optimizer with manually tuned learning rate. Additional details are available in Appendix D. |
| Hardware Specification | No | Our implementation target beanmachine [50] is a declarative PPL selected due to availability of a PPL compiler and support for static analysis plugins. Similar to [4, 46], it uses Py Torch [40] for GPU tensors and automatic differentiation. |
| Software Dependencies | No | To illustrate an implementation of GGA for static analysis, we sketch the operation of the PPL compiler at a high-level and defer to the code in Supplementary Materials for details. A probabilistic program is first inspected using Python s built-in ast module... it uses Py Torch [40] for GPU tensors and automatic differentiation. trained to convergence using the Adam optimizer with manually tuned learning rate. |
| Experiment Setup | No | All experiments are repeated for 100 trials, trained to convergence using the Adam optimizer with manually tuned learning rate. Additional details are available in Appendix D. |