Symbolic Neural Ordinary Differential Equations
Authors: Xin Li, Chengli Zhao, Xue Zhang, Xiaojun Duan
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments In this section, we provide a comprehensive analysis of the performance of our method across experiments with 64GB RAM and NVIDIA Tesla V100 GPU 16GB. To train and test our approach for a given dynamical system s(x, t, u(x, t)), we generate Ntr and Nte trajectories, respectively. Specifically, we utilize a Gaussian random field (GRF) to generate a 1-d parameter function for each track. The experimental results are presented in Figure 2. Our approach consistently outperforms the baselines regarding the prediction error, regardless of the number of points sampled from u(t) by the Deep ONet method. Moreover, our method exhibits a more significant advantage in the extrapolation prediction when the output scale c selected exceeds the training setting (c [0, 10] for training and c = 14 for the extrapolation experiment). This can be attributed to our approach s ability to fully utilize the temporal information of u(t) within the framework of NODEs and learn the inherent relationship between the parameter function u(t) and the system dynamics, thereby enabling highly accurate modeling of the parametric ODEs. ... Comparative Analysis and Ablation Studies For the fair comparison, we conduct a comparative analysis with several standard baselines, namely Deep ONet (Lu, Jin, and Karniadakis 2019), FNO (Li et al. 2020c), PDENET (Long, Lu, and Dong 2019), and MP-PDE (Brandstetter, Worrall, and Welling 2022). ... The experimental results, which are presented in Table 1, demonstrate that our framework is capable of accurately modeling system dynamics even in scenarios where training data is limited and noisy. |
| Researcher Affiliation | Academia | Xin Li 1, Chengli Zhao 1 , Xue Zhang 1 , Xiaojun Duan 1 1College of Science, National University of Defense Technology, Changsha, Hunan 410073, China li EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes methods and learning strategies in prose, without explicitly labeled pseudocode or algorithm blocks. For example, the 'Learning Strategies for SNODEs' section details the three-stage training process but not in a structured pseudocode format. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code for the described methodology, nor does it provide any links to a code repository. |
| Open Datasets | No | To train and test our approach for a given dynamical system s(x, t, u(x, t)), we generate Ntr and Nte trajectories, respectively. Specifically, we utilize a Gaussian random field (GRF) to generate a 1-d parameter function for each track. The GRF is characterized by its mean, denoted by µ, and a radial basis function (RBF) kernel, given by u G n µ, exp h ||t1 t2||2/(2l2) io , (5) where l represents the length scale that determines the smoothness of the sampling function. We then multiply the sampling function by an output scaling factor c to obtain our parameter function u. |
| Dataset Splits | No | To train and test our approach for a given dynamical system s(x, t, u(x, t)), we generate Ntr and Nte trajectories, respectively. Specifically, we utilize a Gaussian random field (GRF) to generate a 1-d parameter function for each track. ... The experimental results, which are presented in Table 1, demonstrate that our framework is capable of accurately modeling system dynamics even in scenarios where training data is limited and noisy. ... Table 1: The prediction MSE ( two standard deviations) under different training set sizes Ntr and noise levels σn. Here, we consider zero-mean Gaussian noise with a standard deviation of σn times the mean absolute value of the training data. |
| Hardware Specification | Yes | Experiments In this section, we provide a comprehensive analysis of the performance of our method across experiments with 64GB RAM and NVIDIA Tesla V100 GPU 16GB. |
| Software Dependencies | No | In the early stages of training, simpler methods such as Euler should be used to expedite training (high-precision methods may cause numerical instability and training failure). In the later stages of training, higher-precision methods such as Runge-Kutta, adaptive-step solvers, can be employed to further train the model and achieve longer-term predictions on the test set. ... Herein, we employ the Euler method for the initial 120 epochs, incorporating progressively increasing prediction steps. Subsequently, for the remaining 80 epochs, the number of prediction steps is maintained at a constant 20, and we switch to the Dopri5 method for further training. |
| Experiment Setup | Yes | Utilizing Eq. (2) directly for training SNODEs encounters several challenges, including the high computational costs and the risk of converging to local optima associated with the classic NODE method, as well as the issue of gradient explosion. Therefore, we propose a three-stage training approach to efficiently model unknown dynamics, as shown in Figure 1. ... Moreover, to enhance the sparsity of the inferred Sym Net network, L1 regularization was incorporated into both L1 and L2, with the regularization coefficient, α, serving as a hyperparameter. During the training process in stages 2 and 3, we employ an adaptive training strategy. To begin, we simply set the prediction steps to 1 and increase it once the training error falls below a predefined threshold. Secondly, we incorporate an adaptive learning rate, which involves adjusting the learning rate at evenly spaced intervals across batches. ... The corresponding outcomes are presented in Figures 4(a)-(b). Here, we take the DR system as an example and provide the training details for our SNODEs framework. In stage 1, the rapid capture of critical components of unknown dynamics is facilitated through flow matching pre-training, with the training error depicted in Figure 4(c), and we obtain ˆF1 = 0.0099sxx + 0.9955u. Herein, the regularization parameter α is set to 0.01, resulting in the training loss exceeding the validation loss. In stage 2, ˆF1 was fine-tuned through the ODESolve prediction, with the corresponding prediction error illustrated in Figure 4(d), culminating in ˆF1 = 0.0098sxx+u 0.0083s+0.0046. Herein, we employ the Euler method for the initial 120 epochs, incorporating progressively increasing prediction steps. Subsequently, for the remaining 80 epochs, the number of prediction steps is maintained at a constant 20, and we switch to the Dopri5 method for further training. |