Node Embeddings via Neighbor Embeddings
Authors: Jan Niklas Böhm, Marius Keute, Alica Guzmán, Sebastian Damrich, Andrew Draganov, Dmitry Kobak
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we show that neighbor-embedding methods are remarkably effective for node-embedding problems and introduce a framework called graph neighbor embeddings (graph NE) (Figure 1). Our work builds on recent literature which allows to effectively optimize neighbor embeddings in high-dimensional embedding spaces (Mc Innes et al., 2018; Damrich et al., 2023). We show that graph NE outperforms Deep Walk and node2vec in terms of local structure preservation, while being conceptually simpler (no random walks are needed) and without requiring costly hyperparameter tuning. Furthermore, we show that graph NE can also be applied for 2D node embeddings (Figure 1), outperforming existing graph-layout methods. In short, our results demonstrate that neighbor embeddings are a powerful approach to graph representation learning that beats state-of-the-art node-embedding algorithms. Sections 5, 6, 7 detail the experimental setup, analysis of learning dynamics, and benchmarking of graph NE against other methods using various datasets and performance metrics. |
| Researcher Affiliation | Academia | Jan Niklas Böhm EMAIL Hertie AI, University of Tübingen, Germany, Marius Keute Hertie AI, University of Tübingen, Germany, Alica Guzmán Hertie AI, University of Tübingen, Germany, Sebastian Damrich EMAIL Hertie AI, University of Tübingen, Germany, Andrew Draganov EMAIL Department of Computer Science, Aarhus University, Denmark, Dmitry Kobak EMAIL Hertie AI, University of Tübingen, Germany |
| Pseudocode | No | The paper includes Python code snippets in Section 4.2 and Section 4.3 to illustrate the usage of CNE and open TSNE libraries, respectively. However, these are direct code examples, not structured pseudocode or algorithm blocks describing the underlying methods in a generalized, language-agnostic format. |
| Open Source Code | Yes | Code availability Our code is available at https://github.com/berenslab/graph-ne-paper. |
| Open Datasets | Yes | We used eight publicly available graph datasets (Table 1). The first five datasets were retrieved from the Deep Graph Library (Wang et al., 2019). The arXiv and MAG dataset were retrieved from the Open Graph Benchmark (Hu et al., 2020). The MNIST k NN dataset was obtained by computing the k NN graph with k = 15 on top of the 50 principal components of the MNIST digit dataset (Le Cun et al., 1998). |
| Dataset Splits | Yes | To calculate k NN accuracy, we randomly split all nodes into a training (90% of all nodes) and a test set (10%), and used the KNeighbors Classifier from scikit-learn (Pedregosa et al., 2011) with k = 15. |
| Hardware Specification | Yes | All computations were performed on a cluster which isolates the computing resources and removes interference between concurrent computations. All 2D experiments require only CPUs and were ran on 8 cores of an Intel Xeon Gold 6226R. Experiments in 128D ran on a single Nvidia 2080ti GPU card. |
| Software Dependencies | No | The paper mentions 'cne version 0.4.0' explicitly. It also refers to 'open TSNE library (Poličar et al., 2024)', 'scikit-learn (Pedregosa et al., 2011)', 'Py Torch Geometric (Fey & Lenssen, 2019)', and 'NetworkX (Hagberg et al., 2008)'. However, apart from 'cne', specific version numbers for these other key software components are not provided within the text. |
| Experiment Setup | Yes | For all experiments with CNE we used the output dimensionality d = 128, following the Deep Walk paper, and the cosine distance (meaning the embedding vectors lie on a hypersphere, Equation 5). We set the batch size to min{8192, |V|/10} (smaller graphs required smaller batch sizes for good convergence) and used full-batch repulsion (m = 2b 2) for better local structure preservation (Damrich et al., 2023). The number of epochs was set to 100. We used the Adam optimizer (Kingma & Ba, 2015) with learning rate 0.001. Graph NE was initialized with 128-dimensional Diffusion Maps (Coifman & Lafon, 2006). The embedding dimensionality d = 2 allows us to use the KL divergence and open TSNE library (Poličar et al., 2024) with default parameters for optimization. It supports Diffusion Maps for initialization (Kobak & Linderman, 2021), sets the learning rate to n to achieve good convergence (Linderman & Steinerberger, 2019; Belkina et al., 2019), and employs the FIt-SNE algorithm (Linderman et al., 2019). In the following experiments, we set the temperature of graph NE with CNE backend to τ = 0.05 for all datasets. For node2vec, we ran a sweep over the parameters p, q {0.25, 0.5, 1, 2, 4}. |