Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Cost-Based Goal Recognition in Navigational Domains
Authors: Peta Masters, Sebastian Sardina
JAIR 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we report our results on the performance of goal recognition in path-planning when using the original (complex) cost difference (RG1), the simpler version (1) that does not reason negatively about observations, and the single-observation version (2). Tests were conducted in a discrete (gridworld) domain using problems adapted from the well-known MOVING-AI pathplanning benchmarks (Sturtevant, 2012), which discretise the underlying maps and groundplans to a 512 512 grid. The aim was to develop an experimental framework for the problem of goal recognition in path-planning to empirically confirm that (i) the case of observations conforming to the only optimal path to a goal (as in Theorem 3) is rare and, otherwise, the simpler formula (1) yields identical posterior probability distributions to formula (RG1); (ii) all three accounts return posterior probability distributions that rank goals the same; and (iii) use of either formula (1) or (2) cuts processing time by more than half. |
| Researcher Affiliation | Academia | Peta Masters EMAIL Sebastian Sardina EMAIL RMIT University, 124 La Trobe St Melbourne, Vic 3000, Australia |
| Pseudocode | No | The paper includes mathematical formulas and definitions of concepts but does not present any structured pseudocode blocks or algorithms labeled as such. The methodologies are described in prose. |
| Open Source Code | No | We used a Python-based infrastructure, originally designed as a simulator and testbed for path-planning algorithms, which already included implementations of A* and Weighted A* in its library (https://tinyurl.com/p4sim). While this indicates the use of a publicly available third-party library, the paper does not explicitly state that the authors' *own* source code for the methodologies described in the paper (e.g., their implementations of costdif1, costdif2, or RMP calculation) is open-sourced. |
| Open Datasets | Yes | Tests were conducted in a discrete (gridworld) domain using problems adapted from the well-known MOVING-AI pathplanning benchmarks (Sturtevant, 2012), which discretise the underlying maps and groundplans to a 512 512 grid. ... from two sets of MOVING-AI benchmarks (Sturtevant, 2012):16 game landscapes from Star Craft; and connected room layouts ... 15. http://movingai.com/ |
| Dataset Splits | Yes | We then extracted observation sequences varying three dimensions: path quality (optimal, suboptimal, greedy), observation density, that is, the proportion of the continuous path extracted to represent the observation sequence (sparse 20%, medium 50%, dense 80%) and two observation strategies (random extracts observations from random locations along the path, prefix extracts a consecutive sequence of location nodes from the start location). |
| Hardware Specification | Yes | New experiments were conducted on a i7 3.4GHz dual core with 10GB RAM in a virtual Linux environment; previous experiments on similar 1.8GHz machine. |
| Software Dependencies | No | We used a Python-based infrastructure, originally designed as a simulator and testbed for path-planning algorithms, which already included implementations of A* and Weighted A* in its library (https://tinyurl.com/p4sim). While Python is mentioned as the language, specific version numbers for Python or any libraries (including A* and Weighted A*) are not provided. |
| Experiment Setup | Yes | For the automated tests, we adopted a value of 0.1 throughout. ... For P2 we compensated for this effect by adding a large constant to the function s output, which raised it always above zero. This significantly reduced the delta. In any event, observe that, whatever the delta, relative rank is always preserved. |