Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks
Authors: HyunGeun Lee, Kijung Yoon
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we use a random graph generator to systematically investigate how the graph size and structural properties affect the predictive performance of GNNs. We present specific evidence that the average node degree is a key feature in determining whether GNNs can generalize to unseen graphs, and that the use of multiple node update functions can improve the generalization performance of GNNs when dealing with graphs of multimodal degree distributions. Accordingly, we propose a multi-module GNN framework that allows the network to adapt flexibly to new graphs by generalizing a single canonical nonlinear transformation over aggregated inputs. Our results show that the multi-module GNNs improve the OOD generalization on a variety of inference tasks in the direction of diverse structural features. |
| Researcher Affiliation | Academia | Hyungeun Lee EMAIL Department of Electronic Engineering Hanyang University Kijung Yoon EMAIL Department of Electronic Engineering Hanyang University |
| Pseudocode | Yes | Algorithm 1 Bounce Grad 1: procedure Bounce Grad(A, Gi=1:N train , Gi=1:N test , SAr, SAf, η, T, α) 2: Initialize SAr, SAf 3: for i 0 to epochmax do 4: T Schedule Temp(T, i/epochmax, α, SAr, SAf) 5: Ai=1:N, SAr, SAf Bounce(Ai=1:N, Gi=1:N train , T, A, Θ, SAr, SAf) 6: Θ Grad(Θ, Ai=1:N, Gi=1:N test , η) |
| Open Source Code | No | The paper does not contain any explicit statements about providing open-source code for the methodology described, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We evaluate the performance of the multi-module GNN framework on the task of approximating marginal inference for general graphs (Pearl, 1988), solving graph theory multi-task problems (Corso et al., 2020), conducting CLRS algorithmic reasoning tasks (Veličković et al., 2022), and benchmarking against OOD scenarios for real-world graphs (Gui et al., 2022). |
| Dataset Splits | Yes | Specifically, we choose five evenly spaced locations on a 2D scatter plot, as shown in Figure 2c. ... The number of training samples in each group may differ slightly; thus, we ensure that the sample count is consistently 1,000 across the five groups Gi=1:5 by providing different input features to the graphs. For the test graphs Gtest, we uniformly sub-sample 1,000 graphs from the entire sample space (N=3.6M) as in Figure 2c (more details in Appendix A.1). ... The size of the training and validation graphs is 16 nodes, while the test graphs have 64 nodes. ... Table A4: Number of graphs in training, ID validation, ID test, and OOD testing sets for 6 graph-level tasks in GOOD benchmark. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for running its experiments. It does not mention specific GPU models, CPU models, or cloud computing specifications. |
| Software Dependencies | No | Table A1: Model hyperparameters for synthetic and real-world graph benchmarks. Table A2: Optimization hyperparameters for synthetic and real-world graph benchmarks. The paper lists software components like 'MPNN', 'GAT', 'MLP', 'GRU', and 'Adam' but does not provide specific version numbers for these or any other software dependencies such as programming languages or libraries (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | Table A2: Optimization hyperparameters for synthetic and real-world graph benchmarks. max epoch 1000, learning rate 0.001, batch size 32, optimizer Adam, weight decay 0.0, loss function KL Divergence, # propagation 10, # samples in meta-testing 5 for Gi (Marginal inference). Similar details are provided for other tasks. |