Boosting Multiple Views for pretrained-based Continual Learning
Authors: Quyen Tran, Tung Lam Tran, Khanh Doan, Toan Tran, Dinh Phung, Khoat Than, Trung Le
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally, our method consistently outperforms state-of-the-art baselines across a wide range of datasets. 5 EXPERIMENTS 5.1 EXPERIMENTAL SETUP Benchmarks. We examine widely used CIL benchmarks, including Split CIFAR-100, Split Image Net R, 5-Datasets, and Split CUB-200 (Please refer Appendix F.1 for more details). Baselines and Metrics. We compare our method with notable methods exploiting pre-trained models for CL scenario... We present two key metrics: the Final Average Accuracy (FAA), denoting the average accuracy after the last task, and the Final Forgetting Measure (FFM)... 5.2 EXPERIMENTAL RESULTS Our approach achieves superior results compared to baselines. Table 1 presents the main results of all the methods. |
| Researcher Affiliation | Collaboration | Quyen Tran1 , Lam Tran1 , Khanh Doan1, Toan Tran1, Dinh Phung3, Khoat Than2 , Trung Le3 1 Qualcomm AI Research 2 Hanoi University of Science and Technology 3 Monash University |
| Pseudocode | Yes | Detailed pseudocode of both algorithms are provided in Algorithm 1. ... D OUR ALGORITHM FOR DIVERSIFYING MULTIPLE VIEWS WITH BOOSTING PRINCIPLE ... Algorithm 2 Multiple-views learning for adapting T tasks sequentially |
| Open Source Code | No | The paper does not contain an explicit statement about the release of source code for the methodology described, nor does it provide a direct link to a code repository. Mentions of 'official code' refer to baselines, not the authors' own implementation. |
| Open Datasets | Yes | F.1 DATASETS We follow the protocols in (Mc Donnell et al., 2023) to construct the following common benchmarks: Split CIFAR-100 Krizhevsky et al. (2009): This dataset comprises images from 100 classes... Split Image Net-R Krizhevsky et al. (2009): This dataset comprises images from 200 classes... 5-Datasets Ebrahimi et al. (2020): This composite dataset incorporates CIFAR-10 Krizhevsky et al. (2009), MNIST Le Cun et al. (1998), Fashion-MNIST Xiao et al. (2017), SVHN Netzer et al. (2011), and not MNIST Bulatov (2011). Split CUB-200 Wah et al. (2011): This dataset consists of fine-grained images of 200 different bird species. |
| Dataset Splits | Yes | F.1 DATASETS We follow the protocols in (Mc Donnell et al., 2023) to construct the following common benchmarks: Split CIFAR-100 Krizhevsky et al. (2009): ...These classes are divided randomly into 10 separate incremental tasks, with each task featuring a distinct set of classes. ... Split Image Net-R Krizhevsky et al. (2009): ...These classes are also randomly divided into 10 distinct incremental tasks. ... Split CUB-200 Wah et al. (2011): ...it is randomly segregated into 10 incremental tasks, each comprising a unique class subset. |
| Hardware Specification | Yes | G.3 COMPLEXITY ANALYSIS (COMPUTATION AND STORAGE COSTS) The reported results were conducted on 2 machines with Tesla V100-SXM2-32GB-LS. |
| Software Dependencies | No | The paper mentions using a 'pre-trained Vision Transformer (Vi T-B/16)' and 'Adam optimizer', but it does not provide specific version numbers for general software dependencies like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | In our method, we set the contrastive regularization weight β = 0.1, the parameter for prompt construction ζ = 0.1, and the confidence threshold for expert filtering is 0.5. The default values for the number of self-improvement steps is 2, the number of expert views is 15, and the dimension of each atomic view is 10,000. ...For the optimization process, we utilized the Adam optimizer, configured with hyper-parameters β1 set to 0.9 and β2 set to 0.999. The training process was conducted using batches of 128 samples, and a fixed learning rate of 0.005 was applied across all models except for CODA-Prompt. For CODA-Prompt, we employed a cosine decaying learning rate strategy, starting at 0.001. |