Constrained Parameter Inference as a Principle for Learning
Authors: Nasir Ahmad, Ellen Schrader, Marcel van Gerven
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To validate COPI as a principle for learning, we compared it against backpropagation by training fullyconnected deep feedforward neural networks trained on the MNIST handwritten digit dataset (Le Cun et al., 2010) and the CIFAR-10 image dataset (Krizhevsky & Hinton, 2009). Figure 1 shows the results of these simulations using learning parameters as described in Appendix B. Table 1: Peak performance (accuracy) measures of COPI vs BP for the results presented in Figure 1. |
| Researcher Affiliation | Academia | Nasir Ahmad EMAIL Department of Artificial Intelligence, Donders Institute, Radboud University Ellen Schrader EMAIL Department of Artificial Intelligence, Donders Institute, Radboud University Marcel van Gerven EMAIL Department of Artificial Intelligence, Donders Institute, Radboud University |
| Pseudocode | Yes | Algorithm 1 Constrained Parameter Inference |
| Open Source Code | Yes | All code used to produce the results in this paper is available at: https://github.com/nasiryahm/Constrained Parameter Inference. |
| Open Datasets | Yes | MNIST handwritten digit dataset (Le Cun et al., 2010) and the CIFAR-10 image dataset (Krizhevsky & Hinton, 2009). |
| Dataset Splits | Yes | To validate COPI as a principle for learning, we compared it against backpropagation by training fullyconnected deep feedforward neural networks trained on the MNIST handwritten digit dataset (Le Cun et al., 2010) and the CIFAR-10 image dataset (Krizhevsky & Hinton, 2009). Figure 1: COPI vs BP performance on standard computer vision classification tasks. A) Train/test accuracy and loss... B) Train/test accuracy... |
| Hardware Specification | No | No specific hardware details are provided in the paper. The paper mentions 'Network execution and training is described in Algorithm 1.' but does not specify the hardware used. |
| Software Dependencies | No | No specific software dependencies with version numbers are provided. The paper discusses parameters for the Adam optimizer but does not list software versions for frameworks like PyTorch or TensorFlow. |
| Experiment Setup | Yes | Training was carried out in mini-batches of size 50 for all simulations (stochastic updates computed within these mini-batches are averaged during application). Table 2: Parameters for CIFAR-10 and MNIST trained networks (cf. Figure 1). All hidden layers used the leaky rectified linear unit (leaky-Re LU) activation function. |