Integral Autoencoder Network for Discretization-Invariant Learning
Authors: Yong Zheng Ong, Zuowei Shen, Haizhao Yang
JMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The proposed IAE-Net is tested with various applications in predictive data science, solving forward and inverse problems in scientific computing, and signal/image processing. Compared with alternatives in the literature, IAE-Net achieves state-of-the-art performance in existing applications and creates a wide range of new applications where existing methods fail. Keywords: Discretization Invariant, Integral Autoencoder, Randomized Data Augmentation, Predictive Data Science, Forward and Inverse Problems, Signal/Image Processing. Section 3 presents the experimental design and results for the applications covered in this paper. |
| Researcher Affiliation | Academia | Yong Zheng Ong EMAIL Department of Mathematics National University of Singapore 21 Lower Kent Ridge Rd, Singapore 119077 Zuowei Shen EMAIL Department of Mathematics National University of Singapore 21 Lower Kent Ridge Rd, Singapore 119077 Haizhao Yang EMAIL Department of Mathematics University of Maryland College Park 4176 Campus Dr., College Park, MD, 20742 |
| Pseudocode | No | The paper describes the architecture and algorithms in prose and mathematical equations within Section 2 'Algorithm Description' and its subsections. It also uses figures to visualize the network structure (e.g., Figure 1, Figure 2, Figure 4). However, there are no explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available on https://github.com/IAE-Net/iae_net. |
| Open Datasets | Yes | The initial conditions to generate this data set, provided by Li et al. (2020c) in https: //github.com/zongyi-li/fourier_neural_operator, is generated according to u0 µ, where µ = N(0, σ2( + τ 2I) α), using τ = 5, σ = τ 2 and α = 2. The code to generate the data set is provided by Li et al. (2020c) in https://github.com/zongyi-li/fourier_neural_ operator. IAE-Net is trained following the data set provided for FNO training by the original paper (Lu et al., 2022) in https://github.com/lu-group/deeponet-fno The data set is obtained from Khoo and Ying (2019). A simple data set illustrating this problem is the ellipses data set obtained from Adler and Oktem (2017) via https://github.com/adler-j/learned_gradient_tomography. For this experiment, the data set used are ECG mixtures obtained from Andreotti et al. (2016) via the link https://physionet.org/content/fecgsyndb/1.0.0. |
| Dataset Splits | Yes | A total of 10000 training samples and 1000 testing samples are generated. A data set of 1000 training data and 100 testing data is generated with ν = 1e 4 and tested with IAE-Net, FNO, FT, GT, and Deep ONet. Further experiments are conducted to test the performance of IAE-Net, FNO and GT for varying degrees of ν, using 1000 training and 100 testing data for each ν = 1e 4, 1e 3, 1e 2, 1e 1 and 1. A total of 1000 training data and 100 testing data obtained from the benchmark data set is used for the experiment. consists of 2000 samples, of which 1900 samples are used as training data and 100 samples are used as test data. For the scattering problem, the model is trained with 10000 s = 81 training data and tested on 1000 testing data for each of the resolutions s = 27, 41, 81, 161, 241 for both the forward and inverse problems. For this experiment, a combined data set is assembled consisting of 10000 data points generated independently for each of the resolutions s = 81, 108, 162, forming a combined total of 30000 data. For this data set, the model is trained with 10000 s = 128 training data and tested on 1000 data for each of the resolutions s = 32, 64, 128, 256, for IAE-Net, FNO, FT, and GT. The models are trained using 25000 s = 2000 length signals, and tested on 6500 testing data for each the resolutions s = 250, 500, 1000, 2000, 4000. |
| Hardware Specification | Yes | The experiments are run with a 48GB Quadro RTX 8000 GPU. |
| Software Dependencies | No | The paper mentions 'existing deep learning software like Pytorch and TensorFlow' and 'Adam optimizer' as well as 'PDE Toolbox in Matlab (The Math Works, 2019)' for ground truth simulation. However, it does not provide specific version numbers for the deep learning frameworks or Python libraries used for the implementation of IAE-Net itself. |
| Experiment Setup | Yes | For the fixed Sz containing the grid points for discretizing the intermediate function u and v in IAE (see Section 2.3.1), we set m = 256, where m denotes the discretization size, thus Sz = {i 1 255 }m i=1. For the 2-d problem, we set m = 64, where m denotes the discretization size along one axis, thus Sz = {(i 1 63 )}, with i, j = 1, . . . , 64. We set w = 64 for both the 1-d and 2-d problems. The data augmentation (DA) strategy proposed in Equation (14) is used by default for IAE-Net, with λ = 1, using interpolation of the training data to the testing sizes. IAE-Net is trained for all the applications with a learning rate of 0.001, using Adam optimizer as the choice of optimizer. A batch size of 50 for 1-d problems, and 5 for 2-d problems, is used, and training is performed for 500 epochs using a learning rate scheduler scaling the learning rate by 0.5 on a plateau with patience of 20 epochs. All the models are trained using the L2 relative error, denoted as L( , ), as the loss function, defined by L(x, y) := ||x y||2 ||y||2 . |