Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Learning Mixtures of Plackett-Luce Models
Authors: Zhibing Zhao, Peter Piech, Lirong Xia
ICML 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that our GMM algorithm is significantly faster than the EMM algorithm by Gormley & Murphy (2008), while achieving competitive statistical efficiency. |
| Researcher Affiliation | Academia | Zhibing Zhao EMAIL Rensselaer Polytechnic Institute, 110 8th St., Troy, NY 12180 USA Peter Piech EMAIL Rensselaer Polytechnic Institute, 110 8th St., Troy, NY 12180 USA Lirong Xia EMAIL Rensselaer Polytechnic Institute, 110 8th St., Troy, NY 12180 USA |
| Pseudocode | Yes | Algorithm 1 GMM for 2-PL Input: Preference profile P with n full rankings. Compute the frequency of each of the 20 moments Compute the output according to (4) |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the methodology is openly available. |
| Open Datasets | No | The paper uses 'synthetic data' that is generated according to a described procedure rather than a publicly available dataset. No access information for a dataset is provided. |
| Dataset Splits | No | The paper does not explicitly provide details about train/validation/test dataset splits. It mentions generating synthetic datasets for evaluation, but not specific partitioning for training, validation, and testing within those datasets. |
| Hardware Specification | Yes | All experiments are run on an Ubuntu Linux server with Intel Xeon E5 v3 CPUs each clocked at 3.50 GHz. |
| Software Dependencies | Yes | The GMM algorithm is implemented in Python 3.4 and termination criteria for the optimization are convergence of the solution and the objective function values to within 10^-10 and 10^-6 respectively. The optimization of (4) uses the fmincon function through the MATLAB Engine for Python. The EMM algorithm is also implemented in Python 3.4 and the E and M steps are repeated together for a fixed number of iterations without convergence criteria. |
| Experiment Setup | Yes | The GMM algorithm is implemented in Python 3.4 and termination criteria for the optimization are convergence of the solution and the objective function values to within 10^-10 and 10^-6 respectively. The EMM algorithm is also implemented in Python 3.4 and the E and M steps are repeated together for a fixed number of iterations without convergence criteria. We have tested all configurations of EMM with 10 and 20 overall MM iterations, respectively. We found that the optimal configurations are EMM-10-1 and EMM-20-1 (shown in Figure 1, results for other configurations are omitted), where EMM-20-1 means 20 iterations of E step, each of which uses 1 MM iteration. |