Universal Graph Continual Learning
Authors: Thanh Duc Hoang, Do Viet Tung, Duy-Hung Nguyen, Bao-Sinh Nguyen, Huy Hoang Nguyen, Hung Le
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We benchmark our method against various continual learning baselines in 8 real-world graph datasets and achieve significant improvement in average performance and forgetting across tasks. |
| Researcher Affiliation | Collaboration | Thanh Duc Hoang , Viet-Tung Do , Duy-Hung Nguyen, Bao-Sinh Nguyen EMAIL Cinnamon AI, Vietnam Huy Hoang Nguyen EMAIL University of Oulu, Finland Hung Le EMAIL Deakin University, Australia |
| Pseudocode | No | The paper describes the proposed method and algorithms in natural language and mathematical equations (e.g., Section 4.1 Graph Experience Replay, Section 4.2 Local-Global Knowledge Distillation) but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about making the source code available, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We benchmark our method against various continual learning baselines in 8 real-world graph datasets...Datasets We consider 3 document image understanding datasets: SROIE (Huang et al., 2019), CORD (Park et al., 2019), and WILDRECEIPT (Sun et al., 2021) (Appendix Table 6)...Datasets We consider 2 molecule graph classification datasets: ENZYMES, and Aromaticity (Xiong et al., 2019) (see details in Appendix Table 6)...Datasets We consider 3 public datasets: Cora Full (Mc Callum et al., 2000), Reddit (Hamilton et al., 2017), and Arxiv1. |
| Dataset Splits | Yes | For CORD, we keep the original train-validation-test split while we further split the training data of SROIE and WILDRECEIPT with a ratio 80-20 for the training and validation set...We split the data into training/validation/test sets with a ratio of 80/10/10. |
| Hardware Specification | No | The paper does not specify any particular hardware used for running the experiments, such as GPU or CPU models. |
| Software Dependencies | No | The paper mentions using a 'simple graph convolutional network (GCN)' and 'Adam optimizer', but it does not specify any software libraries or frameworks with their version numbers (e.g., PyTorch, TensorFlow, Python version). |
| Experiment Setup | Yes | The number of epochs is sampled in the range of [50, 100] with a discretization step of 10, and the learning rate of Adam optimizer is sampled from the set {10 2, 10 3, 10 4, 10 5}...For ER, we set the replay buffer size to 1000 nodes from all classes. For our methods, we optimize the loss weights β and γ in the range of [0.1, 0.9] using a faster hyperparameter search: Bayesian Optimization (Wu et al., 2019) and realize that balance weights β = γ = 0.5 work best. For ER-LS, and ER-GS-LS, we set Kn equal to batch size which is one as graphs in the document are not large and dense compared to other datasets. |