Scalable Generative Modeling of Weighted Graphs
Authors: Richard Williams, Eric Nalisnick, Andrew Holbrook
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulation studies and experiments on a variety of benchmark datasets demonstrate that Bi GG-E best captures distributions over weighted graphs while remaining scalable and computationally efficient. |
| Researcher Affiliation | Academia | Richard Williams EMAIL Department of Biostatistics University of California, Los Angeles Eric Nalisnick EMAIL Department of Computer Science Johns Hopkins University Andrew Holbrook EMAIL Department of Biostatistics University of California, Los Angeles |
| Pseudocode | Yes | Algorithm 1 Bi GG-E Weight Sampling and Embedding Algorithm 2 Adjacency-LSTM Sampling Algorithm |
| Open Source Code | Yes | Our code for Bi GG-E and all comparison models is available at https://github.com/rlwilliams34/Bi GG-E. |
| Open Datasets | Yes | Erdős Rényi: 100 graphs that represent a null case to test whether the models capture the distribution of weighted graphs under an Erdős Rényi model (Erdős & Rényi, 1959). 3D Point Cloud: 41 graphs of household objects (Neumann et al., 2013). Yeast: 100 phylogenetic trees representing the evolutionary history of 154 Saccharomyces Cerevisiae strains inferred from DNA sequence alignments. Edge weights denote evolutionary time along each branch from ancestor to descendant (Hassler et al., 2022). |
| Dataset Splits | Yes | Following the evaluation protocol of Liao et al. (2019), we compare an equal number of graphs generated by each model against a held-out test set. To evaluate scalability on larger graphs, we train on sets of 80 weighted HW-trees with sizes from {100, 200, 0.5K, 1K, 2K, 5K, 10K, 15K}. Each model is trained on a GV100 GPU with 3.2 GB of memory an single precision performance, and we report the MMD on the normalized Laplacian of the resulting weighted graphs against 20 test graphs. |
| Hardware Specification | Yes | Each model is trained on a GV100 GPU with 3.2 GB of memory an single precision performance |
| Software Dependencies | No | The paper mentions software components like 'LSTM cells', 'MLP', 'Tree-LSTM Cell', and 'Adam Optimizer' but does not specify version numbers for any libraries or frameworks used (e.g., Python, PyTorch, TensorFlow, etc.). |
| Experiment Setup | Yes | Hyperparameters For Adj-LSTM, node states were parameterized with a hidden dimension of 128 and use a 2-layer LSTM. An embedding dimension of 32 was used to embed edge existence, and an embedding dimension of 16 was used to embed the weights. Positional encoding was used on the initialized row states. For all models using Bi GG, we use a hidden dimension of 256 with position encoding on the row states, as used in the original Bi GG model. The weight state we use for Bi GG-E has a hidden dimension of 16 for model runs and 32 for scalability runs. Training Procedure Both models were trained using the Adam Optimizer with weight decay and an initial learning rate of 1e 3. The learning rate is decreased to 1e 5 when training loss plateus. Separate plateus are used for the weight parameters and the topology parameters. For the lobster data set, we train Bi GG-E and Bi GG-MLP for 250 epochs and validate at the 100 and 200th epochs. We plateau weight at epoch 50 and topology at epoch 100. We train Adj-LSTM for 300 epochs and validate every 100 epochs. We decay the learning rate at the 25 and 100th epochs. For the HW-tree data set, we train Bi GG-E and Bi GG-MLP for 550 epochs and validate every 100 epoch. We plateu weight at epoch 150 and topology at epoch 200. We train Adj-LSTM for 100 epochs and validate every 25 epochs, where the learning rate is plateaued at epochs 25 and 50. For the Erdős Rényi data set, Bi GG-E was trained for 500 epochs and validated every 250 epochs. We plateu weight at epoch 100 and topology at epoch 500. Due to slow training and poor convergence, the Adjacency-LSTM was only trained for 27 epochs. For the 3D Point Cloud data set, Bi GG-E was trained for 3000 epochs and validated every 1000 epochs. We plateu weight at epoch 500 and topology at epoch 1500. Regularization We use weight decay on the Adam optimizer with decay 1e 4 on topology parameters and 1e 3 on weight parameters. The loss for weight is scaled down by a factor of 10 to balance the topology and weight losses. Upon plateauing both sets of parameters, the scale was increased to a factor of 100 for all graphs except the PT-tree graphs. When both losses were plateaued, the weight loss was updated every other epoch instead of every epoch to allow more fine-tuning of the graph topology without encouraging overtraining on the edge weights. |