Knowledge Localization: Mission Not Accomplished? Enter Query Localization!
Authors: Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct 39 sets of experiments, along with additional visualization experiments, to rigorously confirm our conclusions. [...] This section investigates Q1 and demonstrates the existence of Inconsistent Knowledge (KI). Our experiments adopt GPT-2 (Radford et al., 2019), LLa MA2-7b (Touvron et al., 2023), and LLa MA3-8b (Meta AI, 2024), representing a range of sizes of popular auto-regressive models. |
| Researcher Affiliation | Academia | 1The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China EMAIL, EMAIL |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled 'Pseudocode' or 'Algorithm', nor does it present structured, code-like procedural descriptions. |
| Open Source Code | No | Code is available here. [...] Additionally, we include all data and code we used in the supplementary materials, and the code will be made public after it is compiled. |
| Open Datasets | Yes | Regarding the dataset, we employ the Para Rel dataset (Elazar et al., 2021). For details to the dataset, see Table 5 in Appendix B. |
| Dataset Splits | No | The paper mentions using the Para Rel dataset with 27,610 entries across 36 relations, but it does not specify how this dataset is split into training, validation, or test sets, nor does it provide percentages, counts, or references to predefined splits. |
| Hardware Specification | Yes | Hardware spcification and environment. We ran our experiments on the machine equipped with the following specifications: CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Total CPUs: 56 NVIDIA Ge Force RTX 3090 20. The Standard Memory Config is 24 GB GDDR6X. NVIDIA A100 80GB PCIe 4. The GPU Memory is 80GB HBM2e. |
| Software Dependencies | Yes | Python Version: 3.10.10 Py Torch Version: 2.0.0+cu117 |
| Experiment Setup | Yes | Experimental Hyperparameters of KN Modification In Equation 2, we set λ1 = λ2 = 2. [...] Experimental Hyperparameters of Obtaining Knowledge Synapses In Equations 3 and 4, the scaling factor τ is the same for all three PLMs, with τ = 0.3. [...] Experimental Hyperparameters of Consistency-Aware KN Modification In Equation 5, we set β1 = 0.7 and β2 = 0.3. For the selection of threshold, we consider the dynamic threshold to find the maximum value of CAS, and neurons larger than 0.3 times are selected as KNs. |