Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Detecting danger in gridworlds using Gromov’s Link Condition
Authors: Thomas F Burns, Robert Tang
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Although our main contribution is theoretical, we conduct some small initial experiments to demonstrate the type of information which can be captured in the geometry and topology (see Appendix A.1). To run these experiments, we developed and used a custom Python-based tool (detailed in Appendix A.2). Our focus on small rooms is largely expository, i.e., they are the simplest non-trivial examples illustrating the key features we want to isolate, and naturally reoccur in all larger rooms. Our intention is also to demonstrate a combinatorial explosion in the number of states. We don t recommend constructing the entire state complex in practical applications (indeed, to implement addition of integers on a computer, it is infeasible and unnecessary to construct all integers). Table 1: Data of Gromov s Link Condition failure and commuting 4 cycles in the state complexes of a 3 3 room with varying numbers of agents and no objects. |
| Researcher Affiliation | Academia | Thomas F Burns EMAIL Neural Coding and Brain Computing Unit OIST Graduate University 1919-1 Tancha, Onna-son, Kunigami-gun Okinawa, Japan 904-0495 Robert Tang EMAIL Department of Pure Mathematics Xi an Jiaotong Liverpool University 111 Ren ai Road, Suzhou Industrial Park Suzhou, Jiangsu Province, China 215123 |
| Pseudocode | No | The paper describes methods and processes in narrative text, such as how to compute the state complex and formal definitions of concepts like 'dance'. However, it does not include any clearly labeled pseudocode blocks, algorithms, or structured, code-like procedural steps formatted as an algorithm. |
| Open Source Code | Yes | We developed a Python-based tool for constructing gridworlds with objects and agents. It includes a GUI application for the easy specification of gridworlds and a script which will produce plots and data of the resulting state complex. We ran all experiments on a Lenovo Idea Pad 510-15ISK laptop. The open-source code is available here: https://github.com/tfburns/State-Complexes-of-Gridworlds. |
| Open Datasets | No | The paper focuses on theoretical analysis and conducting experiments on self-generated gridworld configurations. It defines gridworlds as a setting and generates data (e.g., state complexes, link condition failures) based on these configurations for different numbers of agents and room sizes (as shown in Table 1). It does not use or provide concrete access information for any external, publicly available datasets. |
| Dataset Splits | No | The experiments in this paper involve constructing and analyzing state complexes of gridworlds, not training machine learning models on a dataset. Therefore, the concept of training, test, or validation dataset splits is not applicable to the methodology described. |
| Hardware Specification | Yes | We ran all experiments on a Lenovo Idea Pad 510-15ISK laptop. |
| Software Dependencies | No | We developed a Python-based tool for constructing gridworlds with objects and agents. It includes a GUI application for the easy specification of gridworlds and a script which will produce plots and data of the resulting state complex. The text mentions 'Python-based tool' but does not specify a version number for Python or any specific libraries used within the tool. |
| Experiment Setup | No | The paper describes the methodology for constructing state complexes and analyzing their geometric properties. It defines the gridworld parameters (e.g., 3x3 room, varying number of agents) used for the analysis presented in Table 1, but these are environmental configurations rather than experimental hyperparameters (like learning rates, batch sizes, etc.) typically found in machine learning experiment setups. No specific hyperparameters or training configurations are provided. |