Causal Bandits for Linear Structural Equation Models
Authors: Burak Varici, Karthikeyan Shanmugam, Prasanna Sattigeri, Ali Tajer
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper studies the problem of designing an optimal sequence of interventions in a causal graphical model to minimize cumulative regret... Two algorithms are proposed for the frequentist (UCB-based) and Bayesian (Thompson sampling-based) settings... Additionally, a minimax lower of Ω(d L T) is presented... Section 7. Simulations: The main novelty of our work is to use causal graph knowledge without knowing the interventional distributions... We consider linear SEMs with soft interventions under the Bayesian setting. For a given causal graph structure, the prior distributions... are sampled independently at random... We simulate causal graphs under two structural families. Figure 2b compares the cumulative regret of Lin SEM-TS-Gaussian and non-causal UCB algorithms for d = 3 and L = 2. We observe that Lin SEM-TS-Gaussian algorithm significantly outperforms the standard bandit algorithm by exploiting the known causal structure. Next, we look into the graph structure s effect on the regret of Lin SEM-TS-Gaussian. In Figure 2c, we plot the cumulative regrets at T = 5000 for different values of d and L. |
| Researcher Affiliation | Collaboration | Burak Varıcı EMAIL Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute Troy, NY 12180, USA Karthikeyan Shanmugam EMAIL Google Research India Bangalore, India Prasanna Sattigeri EMAIL MIT-IBM Watson AI Lab IBM Research Cambridge, MA 02142, USA Ali Tajer EMAIL Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute Troy, NY 12180, USA |
| Pseudocode | Yes | Algorithm 1 Lin SEM-UCB Algorithm 2 Lin SEM-TS Algorithm 3 Lin SEM-TS-Gaussian |
| Open Source Code | Yes | The codebase for reproducing the simulations are available at https://github.com/bvarici/causal-bandits-linear-sem. |
| Open Datasets | No | Section 7. Simulations: Parameterization. We consider linear SEMs with soft interventions under the Bayesian setting. For a given causal graph structure, the prior distributions of the non-zero entries of observational parameters B are sampled independently at random according to the uniform distribution on [ 1, 0.25] [0.25, 1]. Priors for interventional weights are set as negatives of observational weights, B = B, so that the effect of an intervention is strong. To approximate the Bayesian setting, the true weights [B]i = [B ]i are sampled 10 times from a Gaussian prior with small variances. The distribution of the individual noise terms ϵi is set to N(0, 1) for all nodes. We note that all parameters are unknown to the learner. Simulations for each sampled parameter set are repeated 10 times. We simulate causal graphs under two structural families. |
| Dataset Splits | No | The main experiments involve simulating causal graphs and running bandit algorithms over a time horizon T (e.g., T=5000 iterations), not on a fixed, pre-split dataset. While Appendix D mentions 'We generate 5000 training and 5000 test samples. Then, we perform linear regression to estimate XN from {1, X1, . . . , XN 1} using training data. We use the estimated parameters to predict XN on test data.', this is for a demonstration of non-linearity in the appendix, not for the main experimental validation of the bandit algorithms. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts, or cloud instance types) used for running the simulations or experiments. |
| Software Dependencies | No | The paper does not explicitly list specific software dependencies, such as libraries or frameworks with their version numbers. It only provides a link to a codebase without detailing the software environment required. |
| Experiment Setup | Yes | Section 7. Simulations: Parameterization. We consider linear SEMs with soft interventions under the Bayesian setting. For a given causal graph structure, the prior distributions of the non-zero entries of observational parameters B are sampled independently at random according to the uniform distribution on [ 1, 0.25] [0.25, 1]. Priors for interventional weights are set as negatives of observational weights, B = B, so that the effect of an intervention is strong. To approximate the Bayesian setting, the true weights [B]i = [B ]i are sampled 10 times from a Gaussian prior with small variances. The distribution of the individual noise terms ϵi is set to N(0, 1) for all nodes... Hierarchical Graphs: Each layer ℓ {1, . . . , L} contains d nodes... The total number of nodes is N = d L + 1... Figure 2b compares the cumulative regret of Lin SEM-TS-Gaussian and non-causal UCB algorithms for d = 3 and L = 2. We observe that Lin SEM-TS-Gaussian algorithm significantly outperforms the standard bandit algorithm by exploiting the known causal structure. Next, we look into the graph structure s effect on the regret of Lin SEM-TS-Gaussian. In Figure 2c, we plot the cumulative regrets at T = 5000 for different values of d and L. |