GlycanML: A Multi-Task and Multi-Structure Benchmark for Glycan Machine Learning
Authors: Minghao Xu, Yunteng Geng, Yihang Zhang, Ling Yang, Jian Tang, Wentao Zhang
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The GLYCANML benchmark consists of diverse types of tasks including glycan taxonomy prediction, glycan immunogenicity prediction, glycosylation type prediction, and protein-glycan interaction prediction. Glycans can be represented by both sequences and graphs in GLYCANML, which enables us to extensively evaluate sequence-based models and graph neural networks (GNNs) on benchmark tasks. Furthermore, by concurrently performing eight glycan taxonomy prediction tasks, we introduce the GLYCANML-MTL testbed for multi-task learning (MTL) algorithms. Also, we evaluate how taxonomy prediction can boost other three function prediction tasks by MTL. Experimental results show the superiority of modeling glycans with multi-relational GNNs, and suitable MTL methods can further boost model performance. We provide all datasets and source codes at https://github.com/Glycan ML/Glycan ML and maintain a leaderboard at https://Glycan ML.github.io/project. In Table 2, we report the single-task performance of 18 representative glycan encoders. |
| Researcher Affiliation | Academia | 1Peking University 2Mila Québec AI Institute 3Bio Geometry 4HEC Montréal 5CIFAR AI Research Chair contacts: EMAIL, EMAIL |
| Pseudocode | No | The paper describes model architectures and experimental setups but does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We provide all datasets and source codes at https://github.com/Glycan ML/Glycan ML and maintain a leaderboard at https://Glycan ML.github.io/project. |
| Open Datasets | Yes | We provide all datasets and source codes at https://github.com/Glycan ML/Glycan ML and maintain a leaderboard at https://Glycan ML.github.io/project. We collect the glycans in the Sugar Base database (Bojar et al., 2020b)... We traverse the Gly Connect database (Alocci et al., 2018)... This benchmark dataset is built upon 564,647 protein-glycan interactions deposited in the Lectin Oracle database (Lundstrøm et al., 2022). |
| Dataset Splits | Yes | For each task, we carefully split the benchmark dataset to evaluate the generalization ability of machine learning models in real-world scenarios. For example, in glycan taxonomy prediction, we leave out the glycans with unseen structural motifs during training for validation and test, which simulates the classification of newly discovered glycans in nature with novel molecular structures. we use the motif-based dataset splitting scheme to derive training, validation and test splits with an 8:1:1 ratio. we derive training, validation and test proteins by splitting all clusters with an 8:1:1 ratio. |
| Hardware Specification | Yes | We conduct all experiments on a local server with 100 CPU cores and 4 NVIDIA Ge Force RTX 4090 GPUs (24GB). |
| Software Dependencies | No | Our implementation is based on the Py Torch (Paszke et al., 2019) deep learning library (BSD-style license) and Torch Drug (Zhu et al., 2022) drug discovery platform (Apache-2.0 license). The paper mentions software libraries but does not provide specific version numbers for them. |
| Experiment Setup | Yes | We train with an Adam optimizer (learning rate: 5 10 4, weight decay: 1 10 3) for 50 epochs on taxonomy, immunogenicity and glycosylation type prediction and for 10 epochs on interaction prediction. The batch size is set as 32 for interaction prediction and 256 for other tasks. For model training, we use cross entropy loss to train taxonomy and glycosylation type prediction tasks, use binary cross entropy loss to train immunogenicity prediction, and adopt mean squared error to train interaction prediction. For model selection, 10 times of validation are uniformly performed along the training process, and the checkpoint with the best validation performance is selected for test. For multi-task learning (MTL), the focusing parameter γ of the dynamic task prioritization (DTP) method is set as 2.0, and the model selection of all MTL methods is based on the mean accuracy over all tasks on the validation set. |