Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Unifying Knowledge Base Completion with PU Learning to Mitigate the Observation Bias
Authors: Jonas Schouterden, Jessa Bekker, Jesse Davis, Hendrik Blockeel4137-4145
AAAI 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We aim to empirically answer the following research questions: can we effectively account for observation biases (i.e., obtain more accurate confidence estimates) using the newly proposed propensity-based estimators, (Q1) when the propensities are known, (Q2) when propensities are guessed ( noisy propensities), (Q3) even when the PCA assumption holds? |
| Researcher Affiliation | Academia | KU Leuven, Department of Computer Science, B-3000 Leuven, Belgium Leuven.AI KU Leuven Institute for AI, B-3000 Leuven, Belgium EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our source code is publicly available.5 5https://github.com/ML-KULeuven/KBC-as-PU-Learning |
| Open Datasets | Yes | Our I is the popular KBC benchmark dataset Yago3-10 (Mahdisoltani, Biega, and Suchanek 2015). |
| Dataset Splits | No | The paper does not specify explicit training, validation, and test dataset splits with percentages or sample counts to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper mentions that 'Rules predicting any p P are mined from I with AMIE (Galárraga et al. 2015) with its default settings', but does not provide specific version numbers for software dependencies like programming languages or libraries. |
| Experiment Setup | Yes | Rules predicting any p P are mined from I with AMIE (Galárraga et al. 2015) with its default settings and a minimum CWA(R) 0.1. [...] Propensity scores are required to calculate IPW(-PCA)(R). We use correct propensity scores e( ) for the idealized scenarios in Q1 and Q3 and noisy versions ˆe for Q2. More details about the exact setup can be found in Appendix C. |