Subset Selection Via Implicit Utilitarian Voting
Authors: Ioannis Caragiannis, Swaprava Nath, Ariel D. Procaccia, Nisarg Shah
JAIR 2017 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show that regret-based rules are more compelling than distortion-based rules, leading us to focus on developing a scalable implementation for the optimal (deterministic) regret-based rule. |
| Researcher Affiliation | Academia | Ioannis Caragiannis EMAIL University of Patras, Greece; Swaprava Nath EMAIL Carnegie Mellon University, USA; Ariel D. Procaccia EMAIL Carnegie Mellon University, USA; Nisarg Shah EMAIL Harvard University, USA |
| Pseudocode | No | The paper describes algorithms and derivations using mathematical formulas and text, but no explicit pseudocode or algorithm blocks are provided. For example, Section 5 describes the computation of f reg but not in a pseudocode format. |
| Open Source Code | No | In November 2016, a new social choice website, Robo Vote.org, was launched; some of us have worked on its design and implementation. [...] Based on the results of Sections 4 and 5, we have implemented the deterministic regret minimization rule on Robo Vote. While the paper mentions implementation on Robo Vote.org, it does not provide a direct link to the source code repository for the methodology described in the paper itself. |
| Open Datasets | Yes | Figure 5 shows the results for the second experiment where real-world utility profiles are drawn from one of the Jester datasets (Goldberg, Roeder, Gupta, & Perkins, 2001). Finally, Figure 6 shows the results for the third experiment where real-world preference profiles are drawn from the Sushi dataset (5 000 voters ranking 100 different kinds of sushi) and the T-Shirt dataset (30 voters ranking 11 T-shirt designs) from Pref Lib datasets (Mattei & Walsh, 2013). |
| Dataset Splits | No | For each experiment, we have 8 voters and 10 alternatives, and test for k [4]. For each setting, we perform 10 000 random simulations, and measure both distortion and regret for the actual utility profile, as opposed to the worst-case utility profile. The paper describes the parameters for their simulations and random data generation, but does not provide specific training/test/validation splits for the external datasets used (Jester, Pref Lib) in a way that would allow direct reproduction of the data partitioning for those datasets. |
| Hardware Specification | Yes | The experiments were performed on a single machine with quad-core 2.9 GHz CPU and 32 GB RAM. |
| Software Dependencies | No | We use the SFO toolbox for Matlab (Krause, 2010). The paper mentions the SFO toolbox for Matlab, but does not provide a specific version number for Matlab or the toolbox itself. |
| Experiment Setup | Yes | The running time scales linearly in n, and increases with m k . The experiments were performed on a single machine with quad-core 2.9 GHz CPU and 32 GB RAM. A time limit of 2 minutes was set because a running time greater than this would not be helpful for our website, where the results need to be delivered quickly to the users. While the greedy pruning procedure does help reduce the running time of both the Submodular and Multi ILP approaches, Single ILP still computes f reg much faster than any other approach, solving instances with 50 alternatives in less than 10 seconds. We have therefore implemented Single ILP on Robo Vote. In Section 4, the paper states: 'For each experiment, we have 8 voters and 10 alternatives, and test for k [4]. For each setting, we perform 10 000 random simulations...'. For running time experiments in Section 5: 'over 10 000 instances with n = 15, k = 3, and m varying from 10 to 50'. |