Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
CLIP-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments
Authors: Jessica Dai, Paula Gradu, Christopher Harshaw
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To complement our theoretical results, we conduct simulations using data from a microeconomic experiment. In Section 7, we support these theoretical results with simulations using data from a microeconomic experiment. |
| Researcher Affiliation | Academia | Jessica Dai UC Berkeley EMAIL Paula Gradu UC Berkeley EMAIL Christopher Harshaw MIT EMAIL |
| Pseudocode | Yes | Algorithm 1: CLIP-OGD |
| Open Source Code | Yes | A repository for reproducing simulations is: https://github.com/crharshaw/Clip-OGD-sims |
| Open Datasets | Yes | We evaluate the performance of CLIP-OGD and Explore-then-Commit (ETC) for the purpose of Adaptive Neyman Allocation on the field experiment of Groh and Mc Kenzie [2016], which investigates the effect of macro-insurance on micro-enterprises in post-revolution Egypt. |
| Dataset Splits | No | The paper does not explicitly provide details about train, validation, or test dataset splits. It mentions using "the first T units in the sequence" as the population for a given T, and that "Units are shuffled to appear in an arbitrary order and outcomes are normalized." |
| Hardware Specification | Yes | Simulations were run on a 2019 Mac Book Pro with 2.4 GHz Quad-Core Intel Core i5 and 16 GB LPDDR3 RAM. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python version, library versions) used for the experiments. |
| Experiment Setup | Yes | CLIP-OGD is run with the parameters recommended in Theorem 4.2 and ETC is run with T0 T 1{3 so that the exploration phase grows with T. Theorem 4.2 states parameter values η a 1{T and α a 5 logp Tq. |