Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect
Authors: Ojash Neopane, Aaditya Ramdas, Aarti Singh
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform simulations that demonstrate that our theoretical improvements translate into empirical improvements, especially in the small sample regime, which is critical for applications such as randomized clinical trials. In Section 6, we empirically validate our method, demonstrating its superior performance compared to existing approaches and its competitiveness with often surpassing even an infeasible oracle baseline. In this section, we present experiments to evaluate the empirical performance of our algorithm. We compare OPTrack against the Clip SDT algorithm proposed by Cook et al. (2024), as well as two oracle algorithms that follow the Neyman allocation. We evaluate these algoritmhs on a range of synthetic simulations as well as on a macro-insurance intervention dataset (Groh & Mc Kenzie, 2016). |
| Researcher Affiliation | Academia | 1Machine Learning Department, Carnegie Mellon University 2Department of Statistics & Data Science, Carnegie Mellon University. Correspondence to: Ojash Neopane <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 Optimistic Policy Tracking (OPTrack) 1: for t = 1, 2, . . . do 2: Compute CSt(π ) according to equation (10) 3: Set πt = argminπ CSt(π ) 1 4: Sample At Bernoulli(πt) 5: Observe Rt ν(At) 6: Compute ht according to equation (2) 7: end for |
| Open Source Code | Yes | Code for replicating experiments can be found at the following Git Hub repo: https://github.com/oneopane/ adaptive-ate-estimation. |
| Open Datasets | Yes | We additionally perform experiments on a macro-insurance intervention dataset (Groh & Mc Kenzie, 2016) which investigates the effects of macro-insurance in Egypt. |
| Dataset Splits | No | The paper does not provide explicit training/test/validation dataset splits. It describes generating synthetic data for simulations and running experiments for a range of total rounds (T) and simulations for the 'macro-insurance intervention dataset', but no specific splits are mentioned. For example: 'For each of these problems, we run OPTrack, Clip SDT, and the reward estimation oracle for T {100, 200, ..., 2000} and plot the normalized MSE (T MSE) over 500,000 simulations.' |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments. It mentions 'synthetic simulations' and 'macro-insurance intervention dataset' but no GPU, CPU, or other hardware specifications. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies. It includes a link to a GitHub repository, but no software versions are mentioned in the text. |
| Experiment Setup | Yes | We consider 6 problem instances where both arms follows Bernoulli distributions. For each of these problem instances, we fix the treatment mean to be 1/2 and vary µ0 {0.05, 0.1, 0.2, 0.3, 0.4, 0.5} which in turn leads to problem instances with different Neyman allocations. For each of these problems, we run OPTrack, Clip SDT, and the reward estimation oracle for T {100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1500, 2000} and plot the normalized MSE (T MSE) over 500,000 simulations. |