Efficient Computation of Rankings from Pairwise Comparisons
Authors: M. E. J. Newman
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate this algorithm with applications to a range of example data sets and derive a number of results regarding its convergence. In Section 7 we apply our algorithms to a broad selection of example data, both real and synthetic, finding in every case that the algorithm of this paper is faster than Zermelo s, often by a wide margin. |
| Researcher Affiliation | Academia | M. E. J. Newman EMAIL Center for the Study of Complex Systems University of Michigan Ann Arbor, MI 48109, USA |
| Pseudocode | No | The paper describes iterative algorithms using mathematical equations and textual explanations, such as "Iterating this process, it can be proved subject to certain conditions that we converge to the global maximum of the likelihood..." and "again we choose suitable starting values (for instance at random), then we iterate the form...". It does not contain explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states: 'All empirical data used in this paper are previously published and freely available online.' However, it does not provide any explicit statement about making the source code for the described methodology available, nor does it include a link to a code repository. |
| Open Datasets | Yes | All empirical data used in this paper are previously published and freely available online. Wolves: ...as reported by van Hooffand Wensing (1987). Vervet monkeys: ...as reported by Vilette et al. (2020). American football: ...as compiled by Yurko et al. (2019). Political figures: The results of an online paired comparison survey conducted by the Washington Post newspaper in 2010, ...made available on the survey platform allourideas.org. Photographs: ...study of Pavlichenko and Ustalov (2021)... Soccer: Data from Mart J urisoo at kaggle.com/martj42. School students: ...National Longitudinal Study of Adolescent Health (the Add Health study, Udry et al. 1997)... Information on how to obtain the Add Health data files is available on the Add Health website (https://www.cpc.unc.edu/addhealth). Chess: ...lichess.com during the month of July 2016. The data are from lichess.com via kaggle.com/arevel. |
| Dataset Splits | No | The paper describes generating synthetic data and applying algorithms to full real-world datasets to test convergence. It does not provide specific training/test/validation splits for its own experiments, as the objective is to compute rankings on complete datasets, not to train and evaluate models using such splits. |
| Hardware Specification | No | The paper mentions: 'a single run of Zermelo s algorithm (implemented in the Python programming language on an up-to-date but otherwise unremarkable personal computer circa 2022)'. This description is too general and lacks specific hardware details such as CPU/GPU models or memory specifications. |
| Software Dependencies | No | The paper states the algorithm was 'implemented in the Python programming language'. However, it does not provide specific version numbers for Python or any other software libraries or dependencies used in the experiments. |
| Experiment Setup | No | The paper describes setting 'Initial values of πi ... randomly such that si is drawn from the logistic distribution' and for ties, 'with an initial value of ν = 1 in all cases'. While these are initial conditions for the iterative algorithms, the paper does not provide specific hyperparameters, detailed training configurations, or system-level settings typically found in experimental setup sections for machine learning model training. |