Centrality-guided Pre-training for Graph
Authors: Bin Liang, Shiwei Chen, Lin Gui, HUI WANG, Yue Yu, Ruifeng Xu, Kam-Fai Wong
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on a series of real-world datasets demonstrate that the proposed Cen Pre outperforms the state-of-the-art baselines in the tasks of node classification, link prediction, and graph classification1. |
| Researcher Affiliation | Collaboration | 1The Chinese University of Hong Kong 2Peng Cheng Laboratory 3Harbin Institute of Technology (Shenzhen) 4King s College London 5Ministry of Education Key Laboratory of High Confidence Software Technologies, CUHK EMAIL; EMAIL; EMAIL; EMAIL; EMAIL |
| Pseudocode | No | The paper describes the proposed method using mathematical equations and descriptive text, but it does not contain any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1The source code of this work is released at https://github.com/HITSZ-HLT/CenPre. |
| Open Datasets | Yes | Dataset We evaluate the effectiveness of our proposed Cen Pre framework through the tasks of node classification, link prediction, and graph classification on 13 widely used benchmark datasets. These datasets include the Citation Networks triplet (Kipf & Welling, 2017) (Cora, Citeseer, Pubmed), Amazon Co-Purchase networks (Shchur et al., 2018) (Computer, Photo), TUD Benchmark datasets (Morris et al., 2020) (MUTAG, NCI1, PROTEINS, DD, IMDB-B, RDT-B), and two large-scale graphs from the Open Graph Benchmark (Hu et al., 2020) (ogbn-ar Xiv, ogbl-Collab). |
| Dataset Splits | Yes | For node classification and link prediction tasks, we use a 2-layer GCN (Kipf & Welling, 2017) as the encoder for self-supervised pretraining under the Cen Pre framework, increasing to 4 layers for the ar Xiv (Hu et al., 2020) dataset. After pretraining, we freeze the encoder to extract node embeddings, which are then input into a 2-layer MLP for classification and prediction. Results are reported as mean and standard deviation over 5 runs. For graph classification, we use a GIN (Xu et al., 2019) encoder instead, which is commonly used in previous graph classification works. we use the Adam (Kingma & Ba, 2014) optimizer with an initial learning rate of lr = 0.01. The balancing hyperparameters for loss components are set to λ1 = 1, λ2 = 1, and λ3 = 5, determined through pilot studies. For model parameters, we use grid search to find the optimal parameter combination. For the three hyperparameters λ1, λ2, and λ3, in preliminary experiments, we found that when these three hyperparameters are within a reasonable range of values, the performance of the model will only fluctuate within a certain range. When hyperparameters are set to λ1 = 1, λ2 = 1, and λ3 = 5, the model s performance is optimal. Therefore, we set λ1 = 1, λ2 = 1, and λ3 = 5, also making them to the same scale.Our framework is built on Py Torch (Paszke et al., 2019) and Py Torch Geometric (Fey & Lenssen, 2019), leveraging their datasets and functionalities. Experiments are conducted on an Intel(R) Xeon(R) Gold 6248R CPU at 3.00GHz and an Nvidia Tesla V100 GPU with 32GB VRAM. Table 8 shows all relevant parameter settings for our experiments. We compare our Cen Pre with a series of SOTA baseline models, including 1) supervised learning methods: GCN (Kipf & Welling, 2017), GAT (Velickovic et al., 2018), GIN (Xu et al., 2019), and SAGE (Hamilton et al., 2017); 2) graph kernels methods: WL (Shervashidze et al., 2011) and DGK (Yanardag & Vishwanathan, 2015); 3) self-supervised learning methods: node2vec (Grover & Leskovec, 2016), graph2vec (Narayanan et al., 2017), Info Graph (Sun et al., 2020), GAE (Kipf & Welling, 2016), VGAE (Kipf & Welling, 2016), ARGA (Pan et al., 2018), Graph MAE (Hou et al., 2022), DGI (Veliˇckovi c et al., 2019), GRACE (Zhu et al., 2020), GCA (Zhu et al., 2021b), BGRL (Thakoor et al., 2021), CCA-SSG (Zhang et al., 2021), Graph CL (You et al., 2020), JOAO (You et al., 2021), Info GCL (Xu et al., 2021), Sim GRACE (Xia et al., 2022a), Auto GCL (Yin et al., 2022), Mask GAEe (Li et al., 2023), Mask GAEp (Li et al., 2023), Topo GCL (Chen et al., 2024), Patcher (Ju et al., 2023) and GPA (Zhang et al., 2024). Please refer to Appendix C for detailed information of these baseline models and Appendix D for implementation details, which follows the evaluation protocol established by previous works (Li et al., 2023; Hou et al., 2022). |
| Hardware Specification | Yes | Experiments are conducted on an Intel(R) Xeon(R) Gold 6248R CPU at 3.00GHz and an Nvidia Tesla V100 GPU with 32GB VRAM. |
| Software Dependencies | No | Our framework is built on Py Torch (Paszke et al., 2019) and Py Torch Geometric (Fey & Lenssen, 2019), leveraging their datasets and functionalities. While PyTorch and PyTorch Geometric are mentioned with citations, specific version numbers for these software dependencies are not explicitly provided. |
| Experiment Setup | Yes | For node classification and link prediction tasks, we use a 2-layer GCN (Kipf & Welling, 2017) as the encoder for self-supervised pretraining under the Cen Pre framework, increasing to 4 layers for the ar Xiv (Hu et al., 2020) dataset. After pretraining, we freeze the encoder to extract node embeddings, which are then input into a 2-layer MLP for classification and prediction. Results are reported as mean and standard deviation over 5 runs. For graph classification, we use a GIN (Xu et al., 2019) encoder instead, which is commonly used in previous graph classification works. we use the Adam (Kingma & Ba, 2014) optimizer with an initial learning rate of lr = 0.01. The balancing hyperparameters for loss components are set to λ1 = 1, λ2 = 1, and λ3 = 5, determined through pilot studies. For model parameters, we use grid search to find the optimal parameter combination. For the three hyperparameters λ1, λ2, and λ3, in preliminary experiments, we found that when these three hyperparameters are within a reasonable range of values, the performance of the model will only fluctuate within a certain range. When hyperparameters are set to λ1 = 1, λ2 = 1, and λ3 = 5, the model s performance is optimal. Therefore, we set λ1 = 1, λ2 = 1, and λ3 = 5, also making them to the same scale. Table 8 shows all relevant parameter settings for our experiments. |