Accelerating Batch Active Learning Using Continual Learning Techniques
Authors: Arnav Mohanty Das, Gantavya Bhatt, Megh Manoj Bhalerao, Vianne R. Gao, Rui Yang, Jeff Bilmes
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments across many data domains, including natural language, vision, medical imaging, and computational biology, each with different neural architectures and dataset sizes. CAL consistently provides a 3x reduction in training time, while retaining performance and out-of-distribution robustness, showing its wide applicability. |
| Researcher Affiliation | Academia | University of Washington, Seattle Memorial Sloan Kettering Cancer Center EMAIL |
| Pseudocode | Yes | Algorithm 1 Batch Active Learning; Algorithm 2 Continual Learning; Algorithm 3 The general CAL approach. |
| Open Source Code | No | The paper does not explicitly state that source code for the described methodology is provided, nor does it include a direct link to a code repository. |
| Open Datasets | Yes | FMNIST: The FMNIST dataset consists of 70,000 28 28 grayscale images of fashion items belonging to 10 classes (Xiao et al., 2017). CIFAR-10: CIFAR-10 consists of 60,000 32 32 color images with 10 different categories (Krizhevsky, 2009). Med MNIST: We use the Derma MNIST dataset within the Med MNIST collection (Yang et al., 2021a;b)... Amazon Polarity Review: (Zhang et al., 2015) is an NLP dataset... COLA: COLA (Warstadt et al., 2018) aims to check the linguistic acceptability of a sentence... Single-Cell Cell Type Identity Classification: ...The HCL dataset is ... (Han et al., 2020). |
| Dataset Splits | No | The paper specifies how active learning budgets increase over rounds (e.g., "budgets go from 10% to 50% in increments of 10%") but does not explicitly provide fixed training/test/validation splits for the entire datasets used in the experiments. While validation performance is mentioned, the specific split percentages or counts for validation and test sets are not detailed. |
| Hardware Specification | Yes | A NVIDIA Ge Force RTX 1080 GPU was used to run all the reported experiments. ... Training is done on an NVIDIA Ge Force RTX 2080. ... All reported models were trained on an NVIDIA Ge Force 1080 Ti. ... Training is done on an NVIDIA A100-PCIE-40GB. |
| Software Dependencies | No | The paper mentions using "Huggingface's transformer library" but does not specify a version number for this or any other software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | Hyperparameters: Details about the specific hardware and the choices of hyperparameters used to train models for each technique can be found in Appendix A.5. Appendix A.5: FMNIST: All experiments for FMNIST used a Res Net-18 with an SGD optimizer, with learning rate of 0.01 and batch size of 64. For all the CAL methods, we fix m(h) = 128. ... Similar detailed settings are provided for CIFAR-10, Med MNIST, Amazon Polarity Review, COLA, and Single-Cell Cell-Type Identity Classification. |