Numerical Analysis near Singularities in RBF Networks
Authors: Weili Guo, Haikun Wei, Yew-Soon Ong, Jaime Rubio Hervas, Junsheng Zhao, Hai Wang, Kanjian Zhang
JMLR 2018 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | First, we show the explicit expression of the Fisher information matrix for RBF networks. Second, we demonstrate through numerical simulations that the singularities have a significant impact on the learning dynamics of RBF networks. |
| Researcher Affiliation | Academia | Weili Guo EMAIL Haikun Wei EMAIL Key Laboratory of Measurement and Control of CSE, Ministry of Education, School of Automation Southeast University Nanjing, Jiangsu Province 210096, P.R. China Yew-Soon Ong EMAIL Jaime Rubio Hervas EMAIL School of Computer Science and Engineering Nanyang Technological University, 50 Nanyang Avenue 639798, Singapore Junsheng Zhao EMAIL School of Mathematics Science Liaocheng University Liaocheng, Shandong Province 252059, P.R. China Hai Wang EMAIL Sobey School of Business Saint Mary s University Halifax, Nova Scotia B3H 3C3, Canada Kanjian Zhang EMAIL Key Laboratory of Measurement and Control of CSE, Ministry of Education, School of Automation Southeast University Nanjing, Jiangsu Province 210096, P.R. China |
| Pseudocode | No | The paper describes procedures and methods using mathematical equations and descriptive text, such as "The procedure followed for the numerical analysis is given as: Step 1: ...", but it does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code, direct links to code repositories, or mention of code in supplementary materials. |
| Open Datasets | Yes | Extended Complex Scene Saliency Data set (ECSSD) has been widely used since its release in 2013 (Yan et al., 2013). |
| Dataset Splits | No | In this experiment, we use the method proposed in (Zhang et al., 2014) to extract the features of the images in ECSSD data set as the input of the RBF networks. We get three conspicuity maps in both the rarity and the distinctiveness factors, and one conspicuity map in central bias factor. Thus the number of the nodes in the input layer is 7. The output of the training samples is 1 or 0 , where 1 represents this part of the image is salient and 0 represents this part of the image is not salient. As the distribution of input data is unknown in this experiment, we cannot obtain the analytical form of both ALEs of the training process and the Fisher information matrix. Thus we use batch mode learning to accomplish the experiment. By using a trial-and-error method, we choose the number of hidden unit in the student model to be k = 90 and the spread constant σ = 0.5, such that the student RBF network for the input x is given by: i=1 wiφ(x, Ji). (23) We use 200 samples to train the RBF network. For the learning rate η = 0.002 , the model is trained by the gradient algorithm for 15000 times and the sum squared training error E = 1 i=1 (yi 90 P j=1 wjφ(xi, Jj))2 is used to replace the generalization error. Then we clone it 200 times. Each clone is trained with different random initial weights. The initial student parameters J(0) i and w(0) i are uniformly generated in the interval [ 2, 2]. While the paper mentions using "200 samples to train the RBF network" for the ECSSD dataset, it does not specify how these 200 samples were obtained from a larger dataset (e.g., specific train/test/validation percentages or counts, or a citation to predefined splits) which would be necessary for reproducing the data partitioning. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory, or cloud resources) used for running the experiments. |
| Software Dependencies | No | The paper does not list any specific software components or libraries with their version numbers. |
| Experiment Setup | Yes | We choose the spread constant σ = 0.5. In order to investigate the influence of the singularities in the learning process of RBF networks more accurately, we mainly focus on input x with dimension 1. For the learning rate η = 0.002 , the model is trained by the gradient algorithm for 15000 times and the sum squared training error E = 1 i=1 (yi 90 P j=1 wjφ(xi, Jj))2 is used to replace the generalization error. ... The initial student parameters J(0) i and w(0) i are uniformly generated in the interval [ 2, 2]. |