Neural Feature Learning in Function Space
Authors: Xiangxiang Xu, Lizhong Zheng
JMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To verify the learning algorithms as well as established theoretical properties, we design a series of experiments with various types of data. The main goal is to compare the features learned by neural feature extractors and the corresponding theoretical results. To allow such comparisons, we generate data from given probability distributions of which we know the analytical form of optimal features. The source codes for all experiments are available at github.com/Xiangxiang Xu/NFE, and we defer the implementation details to Appendix D. |
| Researcher Affiliation | Academia | Xiangxiang Xu EMAIL Lizhong Zheng EMAIL Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Cambridge, MA 02139-4307, USA |
| Pseudocode | No | The paper describes methods and procedures using mathematical formulations and prose, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source codes for all experiments are available at github.com/Xiangxiang Xu/NFE, and we defer the implementation details to Appendix D. |
| Open Datasets | No | To allow such comparisons, we generate data from given probability distributions of which we know the analytical form of optimal features. The paper describes generating data for each experiment, for example: "In the experiment, we set |X| = 8, |Y| = 6, and randomly generate a PX,Y PX Y." and "we first generate N = 50 000 sample pairs of X, Y for training". |
| Dataset Splits | Yes | We generate N = 50 000 training sample pairs of X, Y , with instances shown in Figure 17b. We also generate N = 10 000 sample pairs in the same manner, as the testing dataset. |
| Hardware Specification | No | The paper mentions general computation, e.g., "We implement our experiments in Python 3... for neural network training", but does not specify any particular hardware components such as CPU or GPU models, or cloud computing instance types. |
| Software Dependencies | No | The paper mentions software like "Python 3", "PyTorch", "Matplotlib", "NumPy", and "SciPy", but it does not specify explicit version numbers for these software packages or libraries. |
| Experiment Setup | Yes | We apply Adam (Kingma and Ba, 2015) as the optimizer with the default parameters: a learning rate of 10-3, beta1 = 0.9, beta2 = 0.999, and epsilon = 10-8. For each MLP (multilayer perceptron) used in the experiments, we set the activation function to be the softplus function x -> log(1 + ex), which are applied to all layers except the output layer... We train the networks with a minibatch size of 128 for 100 epochs... each MLP is with three layers, where the dimensions for all intermediate features, from input to output, are: input = 1 -> 32 -> 32 -> 2 = output. |