Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
A note on regularised NTK dynamics with an application to PAC-Bayesian training
Authors: Eugenio Clerico, Benjamin Guedj
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We establish explicit dynamics for neural networks whose training objective has a regularising term that constrains the parameters to remain close to their initial value. This keeps the network in a lazy training regime, where the dynamics can be linearised around the initialisation. The standard neural tangent kernel (NTK) governs the evolution during the training in the infinite-width limit, although the regularisation yields an additional term that appears in the differential equation describing the dynamics. This setting provides an appropriate framework to study the evolution of wide networks trained to optimise generalisation objectives such as PAC-Bayes bounds, and hence contribute to a deeper theoretical understanding of such networks. |
| Researcher Affiliation | Academia | Eugenio Clerico EMAIL Universitat Pompeu Fabra, Barcelona Benjamin Guedj EMAIL Centre for AI and Department of Computer Science, University College London & Inria London |
| Pseudocode | No | The paper describes methods using mathematical equations and textual explanations, but it does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code, nor does it provide links to a code repository or mention code in supplementary materials. |
| Open Datasets | No | The paper refers to a theoretical "dataset s made of i.i.d. draws from a distribution µ on Z" and discusses a "binary classification problem (i.e., Y = { 1})" with normalized inputs, but it does not reference any specific, named, publicly available datasets with access information (link, DOI, or formal citation). |
| Dataset Splits | No | The paper is theoretical and does not describe empirical experiments using specific datasets, thus there are no mentions of training/test/validation splits. |
| Hardware Specification | No | The paper focuses on theoretical derivations and does not describe running experiments, therefore no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe running experiments, therefore no specific software dependencies with version numbers are mentioned. |
| Experiment Setup | No | The paper focuses on theoretical derivations and does not describe running experiments, therefore no experimental setup details like hyperparameter values or training configurations are provided. |