Improving physics-informed neural networks with meta-learned optimization
Authors: Alex Bihlo
JMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that the error achievable using physics-informed neural networks for solving differential equations can be substantially reduced when these networks are trained using meta-learned optimization methods rather than using fixed, hand-crafted optimizers as traditionally done. We illustrate meta-trained optimizers for several equations of practical relevance in mathematical physics, including the linear advection equation, Poisson s equation, the Korteweg de Vries equation and Burgers equation. |
| Researcher Affiliation | Academia | Alex Bihlo EMAIL Department of Mathematics and Statistics Memorial University of Newfoundland, St. John s (NL) A1C 5S7, Canada |
| Pseudocode | No | The paper describes the optimization algorithm and training process in textual form and refers to 'Algorithm 2 in Vicol et al. (2021)' for more details, but does not include its own structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The algorithm described here has been implemented using Tensor Flow 2.11 and the codes will be made available on Git Hub1. 1. https://github.com/abihlo/Learnable Optimization Pinns |
| Open Datasets | No | The paper focuses on solving differential equations where the 'data' refers to collocation points sampled from the domain of the problem and comparison is made against analytical or high-resolution numerical reference solutions. It does not use pre-existing, publicly available datasets in the conventional machine learning sense, nor does it provide links or citations to such datasets. |
| Dataset Splits | No | The paper does not use traditional datasets with training/test/validation splits. Instead, it refers to 'collocation points' used for evaluating the loss functions (e.g., '# PDE points 10,000', '# IC/BC points 100' in Table 1) which are generated from the differential equations being solved. The evaluation of results is done by comparing the neural network's solution to an exact or reference numerical solution. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | Yes | The algorithm described here has been implemented using Tensor Flow 2.11 |
| Experiment Setup | Yes | In Table 1, the parameters of the physics-informed neural networks trained are summarized, including '# hidden layers', '# units', '# PDE points', '# IC/BC points', and '# epochs'. Additionally, the text states: 'We use hyperbolic tangents as activation function for all hidden layers. We use mini-batch gradient computation with a total of 10 batches per epoch.' and specifies optimizer settings: 'The learning rate for all standard optimizers was set to η = 5 10 4. For the hyper-parameter tuned version of Adam... The optimal parameters found were then β1 = 0.347, β2 = 0.424 and η = 7.65 10 3. The constants of the learnable optimizer were chosen as λ1 = 5 10 4 and λi = 10 3, i = 2, . . . , 4.' |