MiniFold: Simple, Fast, and Accurate Protein Structure Prediction
Authors: Jeremy Wohlwend, Mateo Reveiz, Matt McPartlon, Axel Feldmann, Wengong Jin, Regina Barzilay
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase the efficacy of our approach by comparing the structure prediction accuracy, inference speed, and memory consumption of Mini Fold against ESMFold in Table 1. We also provide comparisons against Omegafold, and several MSA-based models: Alpha Fold2, Rosetta Fold1, Rosetta Fold2 Jumper et al. (2021); Baek et al. (2021); Lin et al. (2023); Wu et al. (2022). |
| Researcher Affiliation | Academia | Jeremy Wohlwend EMAIL MIT Mateo Reveiz EMAIL MIT Matt Mc Partlon EMAIL University of Chicago Axel Feldmann EMAIL MIT Wengong Jin EMAIL Broad Institute of MIT and Harvard Regina Barzilay EMAIL MIT |
| Pseudocode | Yes | Algorithm 1 outlines the architecture of the mini Former block, which takes as initial input the pairwise representations z RL L c from the embedding module and updates it through triangular multiplicative updates and transition layers. Algorithm 2 outlines the architecture of the transformer-based realizer, which replaces the IPA-based structure module found in Alpha Fold 2. Algorithm 3 outlines the procedure used to realize the distogram coordinates with a paramter-free method. Algorithm 4 Self-gating kernel. Algorithm 5 Feed-forward kernel. |
| Open Source Code | Yes | Our code and trained models are available at https://github.com/jwohlwend/minifold. |
| Open Datasets | Yes | We evaluate Mini Fold using CAMEO targets Haas et al. (2019) released between April 1, 2022 and June 31, 2022, for a total of 191 structures. Similar to ESMFold, we also evaluate on 51 structures from the CASP14 targets... We also use 47 targets from CASP15... Similarly to ESMFold, we constructed a training set from the Alpha Fold Protein Structure database. Varadi et al. (2023). We first cluster Uniref50 sequences Suzek et al. (2007) at 30% sequence similarity... |
| Dataset Splits | Yes | We randomly sample 10000 structure as our fixed validation set for model selection and hyper-parameter optimization. We evaluate Mini Fold using CAMEO targets Haas et al. (2019) released between April 1, 2022 and June 31, 2022, for a total of 191 structures. Similar to ESMFold, we also evaluate on 51 structures from the CASP14 targets, chosen to avoid overlaps with the ESMFold training set. We also use 47 targets from CASP15 with structures made recently publicly available. |
| Hardware Specification | Yes | We train Mini Fold using the same PLM used in ESMFold on a single node of 8x A100 GPU s, and achieve competitive performance... We train the models on a single node using 8x A100 GPU s and a batch size of 16 per GPU for an effective batch size of 128 similar to ESMFold. |
| Software Dependencies | No | We implement these new kernels using the Triton library Tillet et al. (2019)... The table shows latency measurements for 'Torch' and 'Torch-compile' (Table 3 and 4), implying PyTorch, but specific version numbers are not provided. |
| Experiment Setup | Yes | We train Mini Fold using 12 and 48 Miniformer blocks. We train the models on a single node using 8x A100 GPU s and a batch size of 16 per GPU for an effective batch size of 128 similar to ESMFold. We use the Adam optimizer with a learning rate of 3e-5 for the tuned PLM layers, 1e-4 for the structure module parameters and 1e-3 for all others for all parameters. We use a pairwise embedding dimension of 128 in the Miniformer, and a sequence embedding dimension of 1024 with 16 self-attention heads each of 64 dimensions in the structure realizer. Training is performed in two stages: first by limiting to the embedder and mini Former blocks and then by enabling recycling and the structure realizer. The model is trained for about 250K steps during each training stage. |