Named Tensor Notation
Authors: David Chiang, Alexander M Rush, Boaz Barak
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We propose a notation for tensors with named axes, which relieves the author, reader, and future implementers of machine learning models from the burden of keeping track of the order of axes and the purpose of each. The notation makes it easy to lift operations on low-order tensors to higher order ones, for example, from images to minibatches of images, or from an attention mechanism to multiple attention heads. After a brief overview and formal definition of the notation, we illustrate it through several examples from modern machine learning, from building blocks like attention and convolution to full models like Transformers and Le Net. We then discuss differential calculus in our notation and compare with some alternative notations. |
| Researcher Affiliation | Academia | David Chiang University of Notre Dame Alexander M. Rush Cornell University Boaz Barak Harvard University |
| Pseudocode | No | The paper describes mathematical definitions and operations using named tensor notation. It does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | While our notation is inspired by these efforts, our focus is on mathematical notation to be used in papers, whereas previous efforts have focused on code. |
| Open Datasets | No | The paper proposes a mathematical notation and does not conduct experiments requiring specific datasets. Therefore, it does not provide access information for any open datasets used in its own research. |
| Dataset Splits | No | The paper is theoretical and focuses on a mathematical notation. It does not conduct experiments with datasets, and therefore, does not describe any dataset splits. |
| Hardware Specification | No | The paper proposes a mathematical notation and does not include experimental results that would require specific hardware. Therefore, no hardware specifications are provided. |
| Software Dependencies | No | The paper discusses software libraries such as NumPy and PyTorch as contexts for named tensors but does not list specific versioned software dependencies required to replicate any experiments or methodologies described in the paper. |
| Experiment Setup | No | The paper is theoretical, introducing a mathematical notation for named tensors. It does not present any experimental results or describe an experimental setup with hyperparameters. |