Graph-level Representation Learning with Joint-Embedding Predictive Architectures
Authors: Geri Skenderi, Hang Li, Jiliang Tang, Marco Cristani
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through multiple experimental evaluations, we show that Graph-JEPA can learn highly semantic and expressive representations, as shown by the downstream performance in graph classification, regression, and distinguishing non-isomorphic graphs. The experimental section introduces the empirical evaluation of the Graph-JEPA model in terms of downstream performance on different graph datasets and tasks, along with additional studies on the latent space s structure and the encoders parametrization. Furthermore, a series of ablation studies are presented in order to elucidate the design choices behind Graph-JEPA. |
| Researcher Affiliation | Academia | Geri Skenderi EMAIL Bocconi Institute for Data Science and Analytics Bocconi University Hang Li EMAIL Michigan State University Jiliang Tang EMAIL Michigan State University Marco Cristani EMAIL University of Verona |
| Pseudocode | No | The paper describes the methodology using prose and mathematical equations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/geriskenderi/graph-jepa. |
| Open Datasets | Yes | We use the TUD datasets (Morris et al., 2020) as commonly done for graph-level SSL (Suresh et al., 2021; Tan et al., 2023). We utilize seven different graph-classification datasets: PROTEINS, MUTAG, DD, REDDITBINARY, REDDIT-MULTI-5K, IMDB-BINARY, and IMDB-MULTI. ... We use the ZINC dataset for graph regression... We further explore the performance of our model on the synthetic EXP dataset (Abboud et al., 2020)... To gain additional empirical insight into Graph-JEPA and its components, we evaluate the model in two complex long-range tasks from the LRGB benchmark (Dwivedi et al., 2022)... |
| Dataset Splits | Yes | We report the accuracy of ten-fold cross-validation for all classification experiments over five runs (with different seeds). It is worth noting that we retrain the Graph-JEPA model for each fold without ever having access to the testing partition in both the pretraining and fine-tuning stages. We use the ZINC dataset for graph regression and report the Mean Squared Error (MSE) over ten runs (with different seeds), given that the testing partition is already separated. ... For both datasets, we follow the work of Dwivedi et al. (2022) and perform four runs with different seeds. |
| Hardware Specification | Yes | All experiments were performed on Nvidia RTX 3090 GPUs. |
| Software Dependencies | No | The neural network modules were trained using the Adam optimizer (Kingma & Ba, 2014) and implemented using Py Torch (Paszke et al., 2019) and Py Torch-Geometric (Fey & Lenssen, 2019), while the linear classifiers and cross-validation procedure were implemented through the Scikit-Learn library (Pedregosa et al., 2011). No specific version numbers for these libraries are provided in the text. |
| Experiment Setup | Yes | The neural network modules were trained using the Adam optimizer (Kingma & Ba, 2014)... In practice, we use the smooth L1 loss as the distance function... This vector representation is then used to fit a linear model with L2 regularization for the downstream task. Specifically, we employ Logistic Regression with L2 regularization on the classification datasets and Ridge Regression for the ZINC dataset. ... Tables 7 and 8 in the Appendix contain the dataset statistics and the JEPA-specific hyperparameters used in the following experiments, respectively. (Table 8 lists: Num. Subgraphs, Num. GNN Layers, Num. Encoder Blocks, Embedding size, RWSE size, Num. context, Num. targets for various datasets). |