Tensor Decomposition Based Memory-Efficient Incremental Learning
Authors: Yuhang Li, Guoxu Zhou, Zhenhao Huang, Xinqi Chen, Yuning Qiu, Qibin Zhao
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across datasets with varying resolutions consistently demonstrate that our approach substantially boosts the performance of baseline methods, showcasing strong generalization and robustness. |
| Researcher Affiliation | Academia | 1School of Automation, Guangdong University of Technology, Guangzhou, CHINA 2Key Laboratory of Intelligent Detection and the Internet of Things in Manufacturing, Ministry of Education, Guangdong University of Technology, Guangzhou, China 3RIKEN AIP, Tokyo, JAPAN. Correspondence to: Guoxu Zhou <EMAIL>. |
| Pseudocode | No | The paper describes the methodology in prose and uses mathematical equations and diagrams (Fig. 1, Fig. 2) to illustrate concepts, but it does not include a distinct section or figure labeled 'Pseudocode' or 'Algorithm' with structured steps. |
| Open Source Code | No | The paper states: Our implementation is based on the deep learning library PyTorch (Paszke et al., 2019), the CIL toolbox PyCIL (Zhou et al., 2023a), and the tensor development library Tensorly (Kossaifi et al., 2019). However, it does not provide an explicit statement or a link to the source code for the methodology described in this paper. |
| Open Datasets | Yes | Datasets. We conducted experiments on two widely used datasets. CIFAR-100 (Krizhevsky, 2009) is a low-resolution and widely used dataset for image classification tasks. It comprises 60,000 32 32 RGB images categorized into 100 classes... Image Net-100 (Deng et al., 2009) is a subset of the larger Image Net-1000 containing 100 randomly selected categories... Tab. 8 presents results on Tiny-Image Net (M = 2k, 10-tasks setting, i.e., Base 0 Inc 20 ). |
| Dataset Splits | Yes | Protocols. We conduct our experiments following two protocols: learning from scratch (LFS) and learning from half (LFH), which are two different ways to split the classes into incremental phases. LFS: divides all the classes equally in each incremental stage... LFH: the first increment task divides half of the total classes, and the rest are equally divided among the subsequent stages... For example, Base 0, Inc 10 indicates that the LFS protocol is followed, and ten classes are assigned to each incremental task. CIFAR-100 (...) It comprises 60,000 32 32 RGB images categorized into 100 classes, each containing 500 training and 100 testing images. Image Net-100 (...) Each category has approximately 1,300 training samples and 50 test samples... |
| Hardware Specification | No | The paper discusses the software libraries and network architectures used (PyTorch, PyCIL, Tensorly, ResNet-18, ResNet-32) but does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | Our implementation is based on the deep learning library Py Torch (Paszke et al., 2019), the CIL toolbox Py CIL (Zhou et al., 2023a), and the tensor development library Tensorly (Kossaifi et al., 2019). While these software components are mentioned, specific version numbers are not provided. |
| Experiment Setup | Yes | We utilized the Stochastic Gradient Descent (SGD) optimizer for training with the following hyperparameters: Base stage: We used SGD with an initial learning rate of 0.1, momentum of 0.9, and weight decay of 0.0005. The training epoch for all datasets is set to 200 with a batch size of 128. The learning rate is scheduled to decay by 0.1 at epochs 60, 120, and 170. Incremental stage: We maintain the training epoch at 170, with a learning rate and momentum that remain constant concerning the base stage. Yet the learning rate decays with factor 0.1 at epochs 80 and 120, and the weight decay is adjusted to 0.0002. |