Holographic Node Representations: Pre-training Task-Agnostic Node Embeddings
Authors: Beatrice Bevilacqua, Joshua Robinson, Jure Leskovec, Bruno Ribeiro
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally validate our approach, showing that Holo GNN is able to learn representations that transfer between task orders consistently and significantly outperforming the baselines. For instance, on the CORA dataset, the performance of Holo GNN decreases by only 1.5% when pretrained on link prediction and adapted to node classification, while standard GNNs show a 7% performance drop, and link-prediction models such as NBFNet (Zhu et al., 2021) and SEAL (Zhang & Chen, 2018), experience a significantly larger drop of up to 41%. |
| Researcher Affiliation | Academia | Beatrice Bevilacqua Purdue University EMAIL Joshua Robinson Stanford University EMAIL Jure Leskovec Stanford University EMAIL Bruno Ribeiro Purdue University EMAIL |
| Pseudocode | Yes | Algorithm 1 Sequential Breaking... Algorithm 2 Parallel Breaking |
| Open Source Code | Yes | 1Our code is available at https://github.com/beabevi/holognn |
| Open Datasets | Yes | For instance, on the CORA dataset... Planetoid and Movie Lens (Sen et al., 2008; Fey & Lenssen, 2019). We evaluate Holo GNN on Rel Bench (Robinson et al., 2024), a recently proposed benchmark of graph datasets derived from relational databases... molhiv dataset from the OGB benchmark (Hu et al., 2020) |
| Dataset Splits | Yes | We generate train validation and test splits for each task by randomly splitting the edges with a ratio of 80:10:10... For node classification, we employ the standard splits (Yang et al., 2016), while for link prediction, following Zhu et al. (2021), we generate train validation and test splits by randomly splitting the edges with a ratio of 85:5:10... For graph classification, we employ the standard splits (Hu et al., 2020), while for link prediction, we generate train validation and test splits by randomly splitting the edges with a ratio of 85:5:10. |
| Hardware Specification | Yes | We ran our experiments on NVIDIA A100 and Ge Force RTX 4090 GPUs... The exception is that we shrunk the network so as to fit on a 24G GPU. |
| Software Dependencies | Yes | We implemented Holo GNN using Pytorch (Paszke et al., 2019) and Pytorch Geometric (Fey & Lenssen, 2019), and performed hyperparameter tuning using the Weight and Biases framework (Biewald, 2020). |
| Experiment Setup | Yes | We train all models using the Adam optimizer, and report the test for the configuration achieving the best validation metric. Each experiment is repeated for 3 different seeds. Details of hyperparameter grid for each dataset can be found in the following paragraphs... We tuned the learning rate in {0.01, 0.001}... We run our experiments for 1k epochs... We run our experiments for 500 epochs... 10 epochs training for node-level tasks, and 20 for link prediction, 2 GNN message passing layers for ft with sum aggregation, and Adam optimizer with learning rate 0.005 for nodelevel and 0.001 for link-level tasks. The exception is that we shrunk the network so as to fit on a 24G GPU. Namely we used batch size 128, hidden dimension 64, and sampled 64 neighbors per node. |