Amphibian: A Meta-Learning Framework for Rehearsal-Free, Fast Online Continual Learning
Authors: Gobinda Saha, Kaushik Roy
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Amphibian in both task- and class-incremental online continual learning setups (16; 40) on long and diverse sequences of image classification tasks (including Image Net-100) using different network architectures (including Res Net) and achieve better performance in both continual and fast learning metrics compared to the twelve most relevant baselines. |
| Researcher Affiliation | Academia | Gobinda Saha EMAIL Department of Electrical and Computer Engineering Purdue University Kaushik Roy EMAIL Department of Electrical and Computer Engineering Purdue University |
| Pseudocode | Yes | Algorithm 1 Amphibian Algorithm for Online Continual Learning |
| Open Source Code | No | The paper mentions implementing Amphibian in python with specific libraries and states that GPM and SGP are implemented using their respective official open-sourced code repositories, but it does not provide an explicit statement or link for the open-sourcing of Amphibian's code. It also references 'codes from1. https://github.com/GMvande Ven/continual-learning (MIT License)' but this is for EWC, SI, and NCL, not Amphibian. |
| Open Datasets | Yes | We use 5 standard image classification benchmarks in continual learning: 20 tasks split CIFAR-100 (16), 40 tasks split Tiny Imagenet (11), 25 tasks split 5-Datasets (36), 20 tasks split Image Net-100 (47) and 10 tasks split mini Image Net (40). Details on the dataset statistics/splits, and network architectures are provided in the Appendix Section 4.1 and 4.2 respectively. |
| Dataset Splits | Yes | For both split mini Image Net and Image Net-100, 2% training data from each task is kept aside as validation sets. Table A.1: Dataset Statistics. 10% training data from each task is kept aside as validation sets. |
| Hardware Specification | Yes | We ran the codes on a single NVIDIA TITAN Xp GPU (CUDA version 12.1) and reported the results in the paper. |
| Software Dependencies | Yes | We implemented Amphibian in python (version 3.7.6) with pytorch (version 1.5.1) and torchvision (version 0.6.1) libraries. |
| Experiment Setup | Yes | In Amphibian, scale learning rate (η) and initial scale value (λ0 m) hyperparameters were set with grid search (as in (16)) with held out validation data from training sets. Similarly, all the hyperparameters of the baselines were tuned. Details of training setup, implementations and a list of all the hyperparameters considered in the baselines and our method are given in Appendix Section 4. Table A.3: Hyperparameters grid considered for the baselines and Amphibian. The best values are given in parentheses. Here, lr represents the learning rate. All the methods use SGD optimizer unless otherwise stated. The number of epochs in the OCL setup for all methods is 1. ... For all the experiments, except split Image Net-100, a batch size of 10 was used for training. In split Image Net-100 experiments a batch size of 25 samples was used. |