Efficiently Parameterized Neural Metriplectic Systems
Authors: Anthony Gruber, Kookjin Lee, Haksoo Lim, Noseong Park, Nathaniel Trask
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The goal of the following experiments is to show that NMS is effective even when entropic information cannot be observed during training, yielding superior performance when compared to previous methods including GNODE, GFINN, and SPNN discussed in Section 2. The metrics considered for this purpose will be mean absolute error (MAE) and mean squared error (MSE) defined in the usual way as the average ℓ1 (resp. squared ℓ2) error between the discrete states x, x ∈ ℙn×ns. For brevity, many implementation details have been omitted here and can be found in Appendix B. An additional experiment showing the effectiveness of NMS in the presence of both full and partial state information can be found in Appendix D. |
| Researcher Affiliation | Collaboration | Anthony Gruber Center for Computing Research Sandia National Laboratories Albuquerque, NM. USA EMAIL Kookjin Lee School of Computing and Augmented Intelligence Arizona State University Tempe, AZ. USA EMAIL Haksoo Lim Big Data Analytics Laboratory Yonsei University Seoul, Korea EMAIL Noseong Park Big Data Analytics Laboratory Korea Advanced Institute of Science and Technology Daejeon, Korea EMAIL Nathaniel Trask School of Engineering and Applied Science University of Pennsylvania Philadelphia, PA. USA EMAIL |
| Pseudocode | Yes | Algorithm 1 Training neural metriplectic systems 1: Input: snapshot data X ∈ Rn×ns, each column xs = x(ts, µs), target rank r ≥ 1, batch size nb ≥ 1. 2: Initialize networks Atri, B, Kchol, E, S, and loss L = 0 3: for step in Nsteps do 4: Randomly draw batch P = {(tsk, xsk)}nb k=1 5: for (t, x) in P do 6: Evaluate Atri(x), B(x), Kchol(x), E(x), S(x) 7: Automatically differentiate E, S to obtain ∇E(x), ∇S(x) 8: Form A(x) = Atri(x) − Atri(x) and D(x) = Kchol(x)Kchol(x) 9: Build L(x), M(x) according to Lemma 3.2 10: Evaluate ˉx = L(x)∇E(x) + M(x)∇S(x) 11: Randomly draw n1, ..., nl and form tj = t + nj∆t for all j 12: ˉx1, ..., ˉxl = ODEsolve(ˉx, t1, ..., tl) 13: L += l−1 ∑j Loss(xj, ˉxj) 14: end for 15: Rescale L = |P|−1L 16: Update Atri, B, Kchol, E, S through gradient descent on L. 17: end for |
| Open Source Code | Yes | Other required information is provided with the accompanying code. All experiments are conducted on Apple M2 Max chips with 96 GB memory. To provide the mean and the standard deviation, experiments are repeated three times with varying random seeds for all considered methods. |
| Open Datasets | No | For each benchmark problem, a set of trajectories is manufactured given initial conditions by simulating ODEs with known metriplectic structure. The metriplectic forms of these systems are given in the subsections below. For the experiments in Table 2, only the observable variables are used to construct datasets, since entropic information is assumed to be unknown. |
| Dataset Splits | Yes | The resulting 100 trajectories are split into 80/10/10 for training/validation/test sets. |
| Hardware Specification | Yes | All experiments are conducted on Apple M2 Max chips with 96 GB memory. |
| Software Dependencies | Yes | For generating the results reported in Table 2, we implemented the proposed algorithm in Python 3.9.12 and Py Torch 2.0.0. |
| Experiment Setup | Yes | For each compared method, integrating the ODEs is done via the Dormand Prince method (dopri5) Dormand and Prince (1980) with relative tolerance 10 −7 and absolute tolerance 10 −9. The loss is evaluated by measuring the discrepancy between the ground truth observable states xo and the approximate observable states ˉxo in the mean absolute error (MAE) and mean squared error (MSE) metrics, which are defined in the usual way via the ℓ1, ℓ2 norms respectively for true data x(ti) and approximate data ˉx(ti): MAE(ˉx, x) = 1∑i=1 | ˉx(ti) − x(ti)|1, MSE(ˉx, x) = 1∑i=1 | ˉx(ti) − x(ti)|2. The model parameters Θ (i.e., the weights and biases) are updated by using Adamax Kingma and Ba (2014) with an initial learning rate of 0.01. The number of training steps is set as 30,000, and the model parameters resulting in the best performance for the validation set are chosen for testing. |