TLLC: Transfer Learning-based Label Completion for Crowdsourcing
Authors: Wenjun Zhang, Liangxiao Jiang, Chaoqun Li
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on several widely used real-world datasets demonstrate the effectiveness of TLLC. Our codes and datasets are available at https://github.com/jiangliangxiao/TLLC.Section 4 reports the experiments and results. To validate the effectiveness of TLLC, we conduct extensive experiments. This section presents our experiments through three aspects: experimental setup, results, and analysis. ... we conduct experiments on three different real-world datasets: Income, Leaves, and Music genre. ... Figure 2 shows the averaged aggregation accuracy of each label aggregation method after performing label completion by WSLC and TLLC, respectively. ... we independently repeat the experiments on each dataset ten times. ... we directly perform a Friedman test with corresponding post-hoc tests (e.g., Nemenyi test) (Demsar, 2006; Jansen et al., 2023) on each dataset using the results of ten repetitions. ... we conduct an ablation experiment on dataset Income. ... we also perform a parameter sensitivity analysis for it. |
| Researcher Affiliation | Academia | 1School of Computer Science, China University of Geosciences, Wuhan 430074, China 2School of Mathematics and Physics, China University of Geosciences, Wuhan 430074, China. Correspondence to: Liangxiao Jiang <EMAIL>. |
| Pseudocode | Yes | The whole construction process of DS and {Dr T }R r=1 in TLLC is shown in Algorithm 1. ... The whole process of worker modeling in TLLC is shown in Algorithm 2. ... The whole process of label completion in TLLC is shown in Algorithm 3. |
| Open Source Code | Yes | Our codes and datasets are available at https://github.com/jiangliangxiao/TLLC. |
| Open Datasets | Yes | Extensive experiments on several widely used real-world datasets demonstrate the effectiveness of TLLC. Our codes and datasets are available at https://github.com/jiangliangxiao/TLLC. ... we conduct experiments on three different real-world datasets: Income, Leaves, and Music genre. ... We have uploaded these datasets and our codes, which are available at https://github.com/jiangliangxiao/TLLC. |
| Dataset Splits | No | The paper describes the proportion of missing labels in the datasets (e.g., 0.85 for Income), but it does not specify how the datasets are split into training, validation, or test sets for the main experiments. It mentions conducting experiments multiple times for statistical significance and simulating data for sensitivity analysis, but this does not constitute explicit train/test/validation splits for reproducibility of the main results. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, processor types, or memory used for running the experiments. It only specifies software-related parameters and network structure. |
| Software Dependencies | No | The paper states that some methods are implemented on the "Crowd Environment and its Knowledge Analysis (CEKA) (Zhang et al., 2015) platform" and others are implemented "in Python." However, it does not provide specific version numbers for Python, CEKA, or any other libraries or frameworks used, which is necessary for reproducibility. |
| Experiment Setup | Yes | For TLLC, we set K = 2, the number of epochs to Q, and the batch size to 32. Additionally, we set the Siamese network ef in TLLC to a small scale to ensure convergence, and the detailed network structure and parameter settings of ef are shown in Table 2. ... Table 2. Detailed network structure and parameter settings of ef. Layer type: Input layer (Output dimension 128, Activation function ReLU), Fully connected layer (Output dimension 64, Activation function ReLU), Output layer (Output dimension 2, Activation function -). The parameter settings of all existing methods are consistent with those specified in their original papers. |