Depth separation beyond radial functions
Authors: Luca Venturi, Samy Jelassi, Tristan Ozuch, Joan Bruna
JMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | The first contribution of this paper is to extend such results to a more general class of functions, namely functions with piece-wise oscillatory structure, by building on the proof strategy of (Eldan and Shamir, 2016). We complement these results by showing that, if the domain radius and the rate of oscillation of the objective function are constant, then approximation by one-hidden-layer networks holds at a poly(d) rate for any fixed error threshold. The mentioned results show that one-hidden-layer networks fail to approximate high-energy functions whose Fourier representation is spread in the frequency domain, while they succeed at approximating functions having a sparse Fourier representation. |
| Researcher Affiliation | Academia | Luca Venturi EMAIL Courant Institute of Mathematical Sciences New York University New York, NY 10012, USA; Samy Jelassi EMAIL Department of Operations Research and Financial Engineering Princeton University Princeton, NJ 08540, USA; Tristan Ozuch EMAIL Department of Mathematics Massachusetts Institute of Technology Cambridge, MA 02142, USA; Joan Bruna EMAIL Courant Institute of Mathematical Sciences and Center for Data Science New York University New York, NY 10011, USA |
| Pseudocode | No | The paper contains mathematical derivations, theorems (e.g., Theorem 4, 5, 11, 12, 15, 19), propositions (e.g., Proposition 16, 20, 21, 22, 34, 40, 41, 44), and lemmas (e.g., Lemma 3, 6, 25, 26, 27, 28, 29, 30, 31, 32, 33, 38, 39, 42, 43, 45, 46) but no structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states 'License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v23/21-1109.html.' This refers to the license for the paper's content, not the open-sourcing of code related to the described methodology. No specific code repository link or explicit statement about code release for the methodology is provided. |
| Open Datasets | No | The paper is theoretical and focuses on approximation rates and depth separation for neural networks. It does not involve empirical experiments on specific datasets. While it mentions 'data distribution ยต' in a theoretical context (Section 3), it does not refer to any concrete, publicly available datasets. |
| Dataset Splits | No | The paper is purely theoretical, analyzing neural network approximation capabilities through mathematical proofs and propositions. It does not involve empirical experiments or the use of datasets with specified training/test/validation splits. |
| Hardware Specification | No | The paper presents theoretical research, including mathematical derivations and proofs regarding neural network approximation. It does not describe any computational experiments or specify hardware used for such purposes. |
| Software Dependencies | No | The paper is a theoretical work focusing on mathematical analysis of neural networks. It does not describe any implemented systems or experiments that would require specific software dependencies with version numbers. |
| Experiment Setup | No | As a theoretical paper primarily concerned with mathematical proofs and properties of neural networks, it does not include an experimental setup, hyperparameters, or training configurations for empirical validation. |