Unifying Causal Representation Learning with the Invariance Principle
Authors: Dingling Yao, Dario Rancati, Riccardo Cadei, Marco Fumero, Francesco Locatello
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTS This section illustrates the expanded applicability of CRL algorithms under the invariance principle. 5.1 shows improved treatment effect estimation on the high-dimensional causal inference benchmark (Cadei et al., 2024) by enforcing the invariance principle through existing domain generalization techniques (Krueger et al., 2021). This result underscores the practical utility of our unified approach. Additionally, 5.2 provides ablation studies on existing interventional CRL methods (Ahuja et al., 2023; Zhang et al., 2024a), showcasing that the non-trivial distributional invariance required for latent variable identification can arise from non-causal assumptions. Fig. 1 depicts the model performance regarding varying invariance regularization strength λINV. |
| Researcher Affiliation | Academia | Dingling Yao, Dario Rancati, Riccardo Cadei, Marco Fumero, and Francesco Locatello Institute of Science and Technology Austria |
| Pseudocode | No | The paper describes methodologies and theoretical frameworks but does not include any clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | REPRODUCIBILITY STATEMENT... For a brief example of the ecology experiment (5.1), please visit: https://github.com/Causal Learning AI/ISTAnt/blob/main/experiments/invariance.ipynb. |
| Open Datasets | Yes | This experiment focuses on ISTAnt (Cadei et al., 2024), a recent real-world ecological benchmark designed for treatment effect estimation. ... The ISTAnt dataset in 5.1 is published by (Cadei et al., 2024). |
| Dataset Splits | No | Experiment settings. Different videos in ISTAnt are considered different experiments as the experiment settings and treatments vary. We consider hard annotation sampling criteria (more non-annotated than annotated) for both experiments (videos) and positions, as described by Cadei et al. (2024). The paper describes annotation sampling criteria but does not provide specific training/validation/test splits (e.g., percentages or exact counts) for the dataset. |
| Hardware Specification | No | The paper does not explicitly describe any specific hardware used for running its experiments (e.g., specific GPU or CPU models, memory, or cloud computing instance types). |
| Software Dependencies | No | Table 2 mentions 'Encoder DINOv2 (Oquab et al., 2023)', 'Loss BCELoss', and 'Optimizer Adam', but does not provide specific version numbers for software libraries, programming languages (like Python), or frameworks (like PyTorch or TensorFlow). |
| Experiment Setup | Yes | Experiment settings. Different videos in ISTAnt are considered different experiments as the experiment settings and treatments vary. We consider hard annotation sampling criteria (more non-annotated than annotated) for both experiments (videos) and positions, as described by Cadei et al. (2024). For the training, we adopt a domain generalization objective that utilizes the invariance principle (Krueger et al., 2021), which is restated as follows: RV-REx(w g) = λINV Var({R1(w g), . . . , RK(w g)}) | {z } invariance k [K] Rk(w g) | {z } sufficiency we provide a detailed derivation in (f) showing the invariance term above is indeed enforcing risk invariance. We vary the strength of the invariant component in eq. (5.1) by setting the regularization multiplier λINV from 0 (ERM) to 10 000. We repeat 20 independent runs for each λINV to estimate the statistical error. Further implementational details are deferred to App. F.1. Table 2: Model and training details for the case study on ISTAnt (5.1). ... Learning Rate 0.0005, Optimizer Adam (β1 = 0.9, β2 = 0.9, ϵ = 10 8), Batch Size 128, Epochs 15, Seeds range(20). |