CBMA: Improving Conformal Prediction through Bayesian Model Averaging
Authors: Pankaj Bhagwat, Linglong Kong, Bei Jiang
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the performance of our conformal Bayesian model averaging prediction method in two experimental settings: (i) when true model is a part of the model space, and (ii) when true model is not included in the model space. In the former case, our numerical results show that the performance of our methodology is similar to that of conformal Bayes and Bayes predictive sets based on true model. ... The details of our simulation experiments are given in the Appendix A. All codes and results are provided in the supplementary material. |
| Researcher Affiliation | Academia | Pankaj Bhagwat Department of Mathematical and Statistical Sciences, University of Alberta, AB, Canada EMAIL |
| Pseudocode | Yes | Algorithm 1 CBMA: Conformal Bayesian Model averaging |
| Open Source Code | Yes | All codes and results are provided in the supplementary material. |
| Open Datasets | Yes | We demonstrate our method using the California Housing dataset. The dataset is available in sklearn, and consists of n = 20640 subjects |
| Dataset Splits | Yes | We use 40% of the total sample size n as the test set with the remaining 60% forming the training set. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, memory, or detailed computer specifications used for running its experiments. It only mentions execution times without describing the hardware. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | In this experiment, we set the miscoverage level α = 0.20. ... The priors we consider on the parameters are βi N(0, 5), ϵ N+(0, 1), half-normal distribution. ... We set the miscoverage level at α = 0.2. |