A Large-scale Training Paradigm for Graph Generative Models
Authors: Yu Wang, Ryan Rossi, Namyong Park, Huiyuan Chen, Nesreen Ahmed, Puja Trivedi, Franck Dernoncourt, Danai Koutra, Tyler Derr
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate that the pre-trained LGGMs have superior zero-shot generative capability to existing graph generative models. Furthermore, our pre-trained LGGMs can be easily fine-tuned with graphs from target domains and demonstrate even better performance than those directly trained from scratch, behaving as a solid starting point for real-world customization. In this section, we conduct five experiments over the graphs collected from 13 domains to demonstrate the effectiveness of LGGMs in five different aspects. |
| Researcher Affiliation | Collaboration | 1University of Oregon 2Adobe Research 3Cisco AI Research 4Amazon 5University of Michigan 6Vanderbilt University |
| Pseudocode | No | The paper describes methods and processes but does not include any clearly labeled pseudocode or algorithm blocks with structured steps. |
| Open Source Code | Yes | We release the code, the model checkpoint, and the datasets at https://github.com/KINDLab-Fly/LGGM. We follow a rigorous reproducibility routine and provide the code and the dataset necessary to reproduce the results shown in Figures 1, 3, 4, and Tables 2, 4 on our Git Hub repository: https://github.com/KINDLab-Fly/LGGM. |
| Open Datasets | Yes | Recognizing the significant gap in developing LGMs for graph-structured data and their potential revolutionary impact similar to LGMs in other fields, we design the very first large-scale training paradigm that leads to the development of LARGE GRAPH GENERATIVE MODELS (LGGMs) pretrained over 5000 graphs from 13 domains sourcing from the Network Repository (Rossi & Ahmed, 2015; 2016). Table 2 compares the performance of our model, LGGM-X, pre-trained on all graph domains except the held-out domain X, with Di Gress trained on the QM9 dataset. We select TUDataset (Morris et al., 2020), a well-established graph classification dataset consisting of graphs from chemistry and social domains. |
| Dataset Splits | Yes | To ease the introduction of training and evaluation setting in Section 5, we further divide each domain-specific set of graphs Gc into training, validation and testing subsets, notated as Gc = GTrain,c GVal,c GTest,c. We adopt the out-of-distribution evaluation where we iteratively treat each domain X as the unseen one and train the LGGM using training graphs from all other domains, and evaluate its performance on the testing graphs from the unseen domain X. We follow the conventional 10 cross-validation setting for evaluation. |
| Hardware Specification | Yes | All experiments are performed on a machine with A100-80G GPU RAM and 128GB RAM. |
| Software Dependencies | Yes | For Text-to-Graph Generation, the textual encoder used to obtain textual description embeddings is "all-Mini LM-L6-v2". |
| Experiment Setup | Yes | For all experiments, we select the best configuration according to the generation performance on validation graphs and report the final performance on generating testing graphs. We adopt the default hyperparameter settings from Di Gress Vignac et al. (2023) with the following exceptions: we generate 100 graphs per domain for each evaluation and set the training epochs at 300 to ensure convergence. Additionally, we implement gradient accumulation, using a mini-batch size of 12 across 4 accumulations, resulting in an effective batch size of 48. |