Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Semantic Representations of Mathematical Expressions in a Continuous Vector Space
Authors: Neeraj Gangwar, Nickvash Kani
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This work describes an approach for representing mathematical expressions in a continuous vector space. We use the encoder of a sequence-to-sequence architecture, trained on visually different but mathematically equivalent expressions, to generate vector representations (or embeddings). We compare this approach with a structural approach that considers visual layout to embed an expression and show that our proposed approach is better at capturing mathematical semantics. Finally, to expedite future research, we publish a corpus of equivalent transcendental and algebraic expression pairs. |
| Researcher Affiliation | Academia | Neeraj Gangwar EMAIL Electrical and Computer Engineering University of Illinois Urbana-Champaign Nickvash Kani EMAIL Electrical and Computer Engineering University of Illinois Urbana-Champaign |
| Pseudocode | No | The paper describes the proposed approach and methodologies in prose, but does not include any explicit pseudocode blocks or algorithm listings with structured steps. |
| Open Source Code | Yes | The datasets and source code are available on Git Hub. 1https://github.com/mlpgroup/expemb |
| Open Datasets | Yes | Finally, to expedite future research, we publish a corpus of equivalent transcendental and algebraic expression pairs. ... This dataset is available publicly on our project page. ... The datasets and source code are available on Git Hub. 1https://github.com/mlpgroup/expemb |
| Dataset Splits | Yes | Our training set has 4,662,300 input-output pairs comprised of 2,744,824 unique expressions. ... Our validation and test sets contain a single expression per example instead of a pair. ... The validation and test sets contain 2,000 and 5,000 expressions, respectively. Table 1: The number of operators and sequence lengths of training, validation, and test sets of the Equivalent Expressions Dataset. |
| Hardware Specification | Yes | These experiments were run on two 32GB V100 GPUs. ... These experiments were run on one 16GB V100 GPU. |
| Software Dependencies | No | We use the Py Torch implementation of the Transformer architecture for our experiments with a modified version of the decoder to enable caching to speed up the generation process at inference. ... We use Sym Py to generate mathematically equivalent expressions for a given formula. ... Refer to the Sym Py documentation for more details.3https://docs.sympy.org/latest/tutorial/simplification.html 4https://pytorch.org/docs/1.12/generated/torch.nn.Transformer.html |
| Experiment Setup | Yes | We use the Transformer architecture with 6 encoder layers, 6 decoder layers, 8 attention heads, and the Re LU activation. The model dimensions, indicated by Dmodel, are 512 and 64 for the experiments in Sections 5.2 and 5.3, respectively. The layer normalization is performed before other attention and feedforward operations. We use the Adam optimizer (Kingma & Ba, 2015) with a learning rate of 10^-4 and do not consider expressions with more than 256 tokens. Refer to Appendix C for more details. ... We use a fixed seed of 42 for reproducibility, and a label smoothing of 0.1 while computing the loss. For the Equivalent Expressions Dataset, we use a model dimension of 512, a feedforward dimension of 2048, and a batch size of 256. ... We use early stopping with 300K minimum steps, 1M maximum steps, and patience of 30K steps. ... For the Sem Vec datasets, we use a model dimension of 64, a feedforward dimension of 256, and a batch size of 512. ... We use early stopping with 50K minimum steps, 1M maximum steps, and patience of 20K steps or 2 epochs (whichever is smaller). |