FreqMoE: Dynamic Frequency Enhancement for Neural PDE Solvers
Authors: Tianyu Chen, Haoyi Zhou, Ying Li, Hao Wang, Zhenzhe Zhang, Tianchen Zhu, Shanghang Zhang, Jianxin Li
IJCAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on both regular and irregular grid PDEs demonstrate that Freq Mo E achieves up to 16.6 percent accuracy improvement while using merely 2.1 percent parameters (47.32x reduction) compared to dense FNO. Furthermore, the approach demonstrates remarkable stability in long-term predictions and generalizes seamlessly to various FNO variants and grid structures, establishing a new Low frequency Pretraining, High frequency Fine-tuning paradigm for solving PDEs. |
| Researcher Affiliation | Academia | Tianyu Chen1 , Haoyi Zhou2 , Ying Li3 , Hao Wang3 , Zhenzhe Zhang3 , Tianchen Zhu4 , Shanghang Zhang3 , Jianxin Li1 1SKLCCSE, School of Computer Science and Engineering, Beihang University, China 2School of Software, Beihang University, China 3SKLMIP, School of Computer Science, Peking University, China 4School of Reliability and Systems Engineering, Beihang University, China |
| Pseudocode | Yes | Algorithm 1 Sparsely Upcycling of FNO Input: Pretrained FNO Model F, number of experts N, rank r, scaling factor α Output: Upcycled Freq Mo E Model 1: // Initialize expert parameters 2: for each Fourier layer l do 3: R(l) θ F.get pretrained weights(l) 4: Initialize gating network g(l) θ 5: for i in 1 to N do 6: Initialize expert R(l) θi R(l) θ + α A(l) θi B(l) θi 7: end for 8: end for 9: return New Freq Mo E Model F |
| Open Source Code | No | The paper does not provide a direct link to a source code repository or an explicit statement about releasing the code for the described methodology. |
| Open Datasets | Yes | From PDEbench [Takamoto et al., 2022], we choose vortex-dominated flows under Random and Turbulent initializations. [...] Using Geo-FNO s [Li et al., 2023a] challenging scenarios: (1)Airfoil, transonic flows over parameterized NACA-0012 airfoils (Mach 0.8) with shock-induced high frequencies on adapted C-grids ( 200 50). (2)Elasticity, nonlinear material deformations with central voids (radius 0.2-0.4), modeled via 1000 FEM nodes capturing stress concentrations. |
| Dataset Splits | No | The paper mentions using specific datasets but does not explicitly provide training, validation, or test split percentages or sample counts in the main text. |
| Hardware Specification | Yes | The solid lines (left y-axis) show the L2 Relative Error (L2RE) achieved by different numbers of modes, while the dashed lines (right y-axis) represent the corresponding inference time measured on a single NVIDIA V100 (32GB) GPU. |
| Software Dependencies | No | The paper mentions using 'Adam optimizer' but does not specify version numbers for any software libraries, frameworks, or programming languages. |
| Experiment Setup | Yes | Training adopts single-step prediction with Adam optimizer (initial lr=0.001). [...] Both variants employ the IPHI module for coordinate transformation, trained with 50% learning rate decay every 50 epochs. [...] Training stabilizes via expert sparsity loss (factor α = 0.1) with identical hyperparameters to baselines for fair comparison. |