Floating-Point Neural Networks Can Represent Almost All Floating-Point Functions
Authors: Geonho Hwang, Yeachan Park, Wonyeol Lee, Sejun Park
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this work, we study the expressive power of floating-point neural networks, i.e., networks with floating-point parameters and operations. We first observe that for floating-point neural networks to represent all functions from floating-point vectors to floating-point vectors, it is necessary to distinguish different inputs: the first layer of a network should be able to generate different outputs for different inputs. We also prove that such distinguishability is sufficient, along with mild conditions on activation functions. Our result shows that with practical activation functions, floating-point neural networks can represent floating-point functions from a wide domain to all finite or infinite floats. |
| Researcher Affiliation | Academia | 1Department of Mathematical Sciences, GIST 2Department of Mathematics and Statistics, Sejong University 3Department of Computer Science and Engineering, POSTECH 4Department of Artificial Intelligence, Korea University. Correspondence to: Sejun Park <EMAIL>. |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. It describes methodologies using mathematical notation and textual descriptions. |
| Open Source Code | No | The paper does not contain any explicit statements about providing open-source code or links to code repositories for the described methodology. |
| Open Datasets | No | This paper is theoretical and does not utilize any specific datasets for experiments. It focuses on the expressive power of floating-point neural networks for representing functions over domains of floating-point numbers. |
| Dataset Splits | No | This paper is theoretical and does not involve experiments using datasets, thus no dataset splits are mentioned. |
| Hardware Specification | No | The paper is purely theoretical and does not describe any experiments that would require specific hardware. Therefore, no hardware specifications are provided. |
| Software Dependencies | No | The paper focuses on theoretical proofs and does not mention any specific software dependencies with version numbers that would be required to replicate experimental results. |
| Experiment Setup | No | The paper is theoretical and does not describe any experimental setup, hyperparameters, or system-level training settings. |