POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfinding
Authors: Alexey Skrynnik, Anton Andreychuk, Anatolii Borzilov, Alexander Chernyavskiy, Konstantin Yakovlev, Aleksandr Panov
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The results of this comparison, which involves a variety of state-of-the-art MARL, search-based, and hybrid methods, are presented. (Abstract) |
| Researcher Affiliation | Collaboration | Alexey Skrynnik B1,2,3, Anton Andreychuk B1, Anatolii Borzilov2,3, Alexander Chernyavskiy3, Konstantin Yakovlev2,1, and Aleksandr Panov1,3 1AIRI, Moscow, Russia; 2FRC CSC RAS, Moscow, Russia; 3MIPT, Dolgoprudny, Russia |
| Pseudocode | No | The paper describes existing algorithms and baselines, but does not include any clearly labeled pseudocode or algorithm blocks for a new method. |
| Open Source Code | Yes | The source code is available at: POGEMA Benchmark2, POGEMA Toolbox3 and POGEMA Environment4. 2https://github.com/Cognitive-AI-Systems/pogema-benchmark 3https://github.com/Cognitive-AI-Systems/pogema-toolbox 4https://github.com/Cognitive-AI-Systems/pogema |
| Open Datasets | Yes | Cities a set of city maps from Moving AI the existing benchmark widely used in heuristic-search community Sturtevant (2012). The contained maps have a varying structure and 256 256 size. It can be used to show how the approach deals with single-agent pathfinding and also deals with the maps that have out-of-distribution structure. |
| Dataset Splits | Yes | POGEMA benchmark contains 6 different sets of maps and all baseline approaches were evaluated on them either on MAPF or on LMAPF instances. Regardless the type of instances, number of maps, seeds and agents were the same. Table 3 contains all information about these numbers. ... Start and goal locations are generated via random generators. They are generated with fixed seeds, thus can be reproduced. |
| Hardware Specification | Yes | To evaluate all the presented approaches integrated with POGEMA we have used two workstations with equal configuration, that includes 2 NVidia Titan V GPU, AMD Ryzen Threadripper 3970X CPU and 256 GB RAM. ... Appendix L: POGEMA SPEED PERFORMANCE EVALUATION. ...AMD Ryzen Threadripper 3970X 32-Core Processor, an older server-side Intel(R) Xeon(R) CPU @ 2.20GHz, ...and the Apple Silicon M1... |
| Software Dependencies | No | The paper mentions using 'Petting Zoo', 'Py MARL', 'Gymnasium', 'Py MARL2 framework', and 'Wandb framework', but does not provide specific version numbers for any of these software components. |
| Experiment Setup | Yes | Table 9: Best hyperparameters found by hyperparameter sweep, which is different from defaults ones. Hyperparameter IQL QPLEX QMIX VDN Batch Size 64 64 32 32 Learning Rate 0.002 0.002 0.001 0.001 RNN Size 128 128 256 256. ... We adjusted the training maps to be approximately 16 16, which proved to be more effective and populated them with 8 agents. ... We continued training the approaches until they reached a plateau, which for most algorithms is under 1 million steps. |