Sparse Additive Gaussian Process Regression
Authors: Hengrui Luo, Giovanni Nattino, Matthew T. Pratola
JMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compare our model against popular alternatives on simulated and real datasets, and find the performance is competitive, while the fully Bayesian procedure enables the quantification of model uncertainties. Keywords: Sparse Gaussian Process, Recursive Partition Scheme, Bayesian Additive Model, Nonparametric Regression. ... 4. Simulation Study ... 5. Real Data Applications |
| Researcher Affiliation | Academia | Hengrui Luo EMAIL Department of Statistics The Ohio State University Columbus, OH 43210, USA Giovanni Nattino EMAIL Division of Biostatistics, College of Public Health The Ohio State University Columbus, OH 43210, USA Matthew T. Pratola EMAIL Department of Statistics The Ohio State University Columbus, OH 43210, USA |
| Pseudocode | Yes | Algorithm 1 Pruning algorithm for RP scheme. ... Algorithm 2 Sampling pseudo-inputs given RP scheme BN. ... Algorithm 3 MCMC algorithm for SAGP model. |
| Open Source Code | No | The paper uses implementations of third-party models like 'SGP regression. ... use the implementation of SGP in the Matlab package implementation SPGP at http: //www.gaussianprocess.org accompanying the paper by Snelson and Ghahramani (2006).' and 'Bayesian Additive Regression Trees (BART) ... and the implementation at http://bitbucket.org/mpratola/openbt.'. However, there is no explicit statement or link provided for the open-sourcing of the Sparse Additive Gaussian Process (SAGP) model developed in this paper. |
| Open Datasets | Yes | Temperature data: n = 247, d = 2; (https://www.image.ucar.edu/Data/US.monthly.met/USmonthly Met.shtml, US precipitation and temperature (1895-1997) dataset). ... Ice Sheet data: n = 2,226, d = 2; (Blankenship et al., 2004). ... U.S. Antarctic Program (USAP) Data Center, 2004. doi: doi:10.7265/ N5WW7FKC. URL https://www.usap-dc.org/view/dataset/609099. |
| Dataset Splits | Yes | The data are split into training and testing sets, with sizes 150 and 50, respectively. ... The performance is evaluated quantitatively with the out-of-sample RMSE, coverage of 95% PIs and average interval score on a 25% test set. ... we use 10-fold cross-validation to select the number of layers L |
| Hardware Specification | Yes | With respect to the computation time, setting the burn-in size to 10,000 and the size of posterior samples to 1,000, an SAGP model can be fit on one dataset of size n = 200 in 3 to 5 minutes, depending on the configuration of m and L, using a laptop with an Intel Core-i5 2.30GHz processor. The time that was needed to fit the model on one batch of 1,000 simulated datasets are summarized in Table 1. on a 40-core cluster. |
| Software Dependencies | Yes | We use the implementation of GP regression model in the R package Dice Kriging by Roustant et al. (2012). ... Bayesian Additive Regression Trees (BART) Chipman et al. (1998, 2010); Pratola et al. (2020). We used the default number of trees as specified in Chipman et al. (2010) and the implementation at http://bitbucket.org/mpratola/openbt. ... BCART) (Chipman et al., 1998), implemented in the Bayes Tree package on CRAN (version 0.3-1.4). We also considered two frequentist models: full GP and Local Approximate GP (la GP) (Gramacy et al., 2016), implemented in the la GP package on CRAN (version 1.5-5). |
| Experiment Setup | Yes | In each dataset, we fit the SAGP model with three configurations of m, L: (i) m = 5, L = 4; (ii) m = 10, L = 3; (iii) m = 15, L = 3. ... For each generated dataset, the models are fit on the training data and used to predict the response on the testing data. ... In our specific implementation, we discard the first 10,000 samples as burn-in and keep the following 1,000 samples to compute posterior estimates. ... For GP regression we used MLE estimates with the Matern(5/2) kernel. For BART we use the default settings (Chipman et al., 2010). For SAGP, we choose L = 3, m = 25 and calibrate the α, β of the noise prior in SAGP and the noise estimate in BART according to MLE of noise estimate from GP. |