Metamizer: A Versatile Neural Optimizer for Fast and Accurate Physics Simulations
Authors: Nils Wandel, Stefan Schulz, Reinhard Klein
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the very same network (same architecture and same weights) across a wide range of linear and nonlinear PDEs and showcase its generalization capabilites even to PDEs that were not covered during training. |
| Researcher Affiliation | Academia | Nils Wandel, Stefan Schulz & Reinhard Klein Department of Computer Science University of Bonn 53115 Bonn, Germany EMAIL |
| Pseudocode | No | The paper describes the Metamizer architecture and training cycle in detail, but it does not present any explicitly labeled pseudocode or algorithm blocks. The processes are described in prose and through diagrams. |
| Open Source Code | Yes | Code as well as a pretrained model are available on Github. |
| Open Datasets | No | Metamizer was trained without any training data but directly on the mean square residuals of PDEs such as the Laplace, advection-diffusion or Navier-Stokes equations and cloth simulations in a meta-learning manner. ... Note that in contrast to data-driven methods, no pre-generated ground truth data is needed. ... Furthermore, since this way the training pool constantly generates new training data on the fly, its memory footprint is small enough such that the entire training pool can be kept in GPU memory. |
| Dataset Splits | No | The paper states that "no pre-generated ground truth data is needed" and that "the training pool constantly generates new training data on the fly." Therefore, there are no traditional datasets with specified training/validation/test splits. |
| Hardware Specification | Yes | After training for about 6 hours on a Nvidia Geforce RTX 4090, we obtained a single model that was able to solve various PDEs at high precision and produced all of the results shown in the following section. ... We run Metamizer against various GPU based solvers of the Cu Py package on the same Nvidia Geforce RTX 4090. |
| Software Dependencies | No | Pytorch libraries such as higher (Grefenstette et al., 2019) or learn2learn (Arnold et al., 2020) allow for example to automatically optimize hyperparameters of optimizers like Adam through gradient descent. ... We compared our approach to several gradient descent based methods of the pytorch optimizer package (SGD, Adam, Adam W, RMSprop, Adagrad, Adadelta) and iterative sparse linear system solvers from Cu Py (conjugate gradients, minres, gmres, lsmr). |
| Experiment Setup | Yes | Then, the step size can be carefully increased. ... Metamizer increases si to lower L more efficiently. ... After training for about 6 hours on a Nvidia Geforce RTX 4090, we obtained a single model that was able to solve various PDEs at high precision... We compared our approach to several gradient descent based methods of the pytorch optimizer package (SGD, Adam, Adam W, RMSprop, Adagrad, Adadelta) and iterative sparse linear system solvers from Cu Py (conjugate gradients, minres, gmres, lsmr). Figure 6 shows that traditional gradient based approaches are not competitive with current sparse linear system solvers regarding runtime or accuracy. Furthermore, these gradient based optimizers require tuning of hyperparameters such as learning rate or momentum (more information is provided in Appendix E). ... By iterating this training cycle, the training pool becomes filled with more and more realistic training data and the neural optimizer becomes better and better at minimizing physics-based losses. ... This recomputed loss is then used to optimize the Metamizer with Adam (lr=0.001). ... For the initial state, we set u0 = u L0 = u0 = 0 and set s0 = 0.05 to avoid initial divergence by making only small steps at the beginning. ... For every simulation time step, we perform a certain number of optimization steps to reduce L. By choosing the right number of iterations per time step we can make a trade-off between speed and accuracy. Figure 5 b) shows how Metamizer automatically adjusts the scaling si when simulating the advection-diffusion equation at 10 iterations per time step. ... Table 1 shows accuracy levels for 1, 5, 20 and 100 iterations per timestep. |