Continual Learning by Modeling Intra-Class Variation
Authors: Longhui Yu, Tianyang Hu, Lanqing HONG, Zhen Liu, Adrian Weller, Weiyang Liu
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we perform empirical studies which demonstrate that our method, as a simple plug-and-play component, can consistently improve a number of memory-based continual learning methods by a large margin. |
| Researcher Affiliation | Collaboration | Longhui Yu EMAIL School of ECE, Peking University Tianyang Hu, Lanqing Hong* EMAIL Huawei Noah s Ark Lab Zhen Liu EMAIL Mila, Université de Montréal Adrian Weller EMAIL University of Cambridge and The Alan Turing Institute Weiyang Liu* EMAIL University of Cambridge and Max Planck Institute for Intelligent Systems Tübingen |
| Pseudocode | Yes | Algorithm 1 Weight-Adversarial Perturbation |
| Open Source Code | Yes | The code is made publicly available at https://github.com/yulonghui/MOCA. |
| Open Datasets | Yes | In this section, we evaluate existing competitive baselines and different MOCA variants on CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009) and Tiny Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | For offline and online continual learning, we divide the dataset into five tasks for CIFAR-10 (two classes per task) and CIFAR-100 (20 classes per task), and divide the dataset into 10 tasks for Tiny Image Net (20 classes per task). |
| Hardware Specification | No | No specific hardware details (like exact GPU/CPU models or memory amounts) are provided for running the experiments. The acknowledgements mention 'MindSpore, CANN (Compute Architecture for Neural Networks) and Ascend AI Processor' but without specific model numbers for actual experimental hardware. |
| Software Dependencies | No | The paper mentions 'PyTorch implementation' in Appendix A, but does not specify any version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | For offline continual learning... the perturbation magnitude λ is set as 2.0... the dropout rate is set as 0.5... proxy loss weight is set as 10... training epoch is set as 50. The batch size is set as 32 and the initial learning rate is est as 0.1. For online continual learning... λ is set as 0.8. The batch size is set as 10. The initial learning rate is est as 0.1. For proxy-based continual learning... λ is set as 1.0. |