MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy
Authors: Haitian Jiang, Renjie Liu, Zengfeng Huang, Yichuan Wang, Xiao Yan, Zhenkun Cai, Minjie Wang, David Wipf
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size. Our source code is available at https://github.com/haitian-jiang/Muse GNN. [...] Finally, in Section 6 we provide complementary empirical support that Muse GNN performance is stable in practice, preserving competitive accuracy and scalability across task size. En route, we achieve SOTA performance w.r.t. homogeneous graph models applied to the largest, publiclyavailable node classification datasets from OGB and IGB exceeding 1TB in size. |
| Researcher Affiliation | Collaboration | Haitian Jiang1 Renjie Liu2 Zengfeng Hunag3 Yichuan Wang4 Xiao Yan5 Zhenkun Cai6 Minjie Wang6 David Wipf6 1New York University 2Southern University of Science and Technology 3Fudan University 4UC Berkeley 5Centre for Perceptual and Interactive Intelligence 6Amazon Web Services |
| Pseudocode | Yes | Algorithm 1 Muse GNN Training Procedure |
| Open Source Code | Yes | Our source code is available at https://github.com/haitian-jiang/Muse GNN. |
| Open Datasets | Yes | We evaluate the performance of Muse GNN on node classification tasks from the Open Graph Benchmark (OGB) (Hu et al., 2020; 2021) and the Illinois Graph Benchmark (IGB) (Khatua et al., 2023), which are based on homogeneous graphs spanning a wide range of sizes. |
| Dataset Splits | Yes | In the presented results, the training, validation and test set splits all follow the original splits from the datasets. |
| Hardware Specification | Yes | We use a single AWS EC2 p4d.24xlarge instance to run the speed experiments. It comes with dual Intel Xeon Platinum 8275CL CPU (48 cores, 96 threads), 1.1TB main memory and 8 A100 (40GB) GPUs. |
| Software Dependencies | Yes | All the models and experiments are implemented in Py Torch (Paszke et al., 2019) using the Deep Graph Library (DGL) (Wang, 2019). The parameters are optimized by Adam optimizer (Kingma & Ba, 2014). |
| Experiment Setup | Yes | For all experiments, we choose the following fixed Muse GNN settings: Both f(X; W) and g(Y ; θ) are 3-layer MLPs, the number of unfolded layers K is 8 (rationale discussed later below), and the embedding dimension d is 512, and the forgetting factor ρ for the online mean estimation is 0.9. [...] In the training process, we set the dropout rate of the MLP layers to be 0.2, and we do not have dropout between the propagation layers. The parameters are optimized by Adam optimizer (Kingma & Ba, 2014) with the weight decay parameter set to 0 and the learning rate being 0.001. For ogbn-arxiv and ogbn-papers100M, α = 0.05, λ = 20. For MAG240M and the IGB series datasets, α = 0.2, λ = 4. And for ogbn-products, α = λ = 1 with preconditioning on the degree matrix for each unfolded step/layer. The batch size n s in the offline samples are all set to 1000. |