Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
HLMEA: Unsupervised Entity Alignment Based on Hybrid Language Models
Authors: Xiongnan Jin, Zhilin Wang, Jinpeng Chen, Liu Yang, Byungkook Oh, Seung-won Hwang, Jianqiang Li
AAAI 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have conducted experiments on benchmark datasets to verify the effectiveness of the proposed HLMEA. The research questions that we aim to answer are listed as follows: (Q1) Does HLMEA outperform state-of-the-art (SOTA) unsupervised EA methods on the benchmark datasets? ... We conduct extensive experiments on benchmark datasets, and the results demonstrate that HLMEA significantly outperforms unsupervised and even supervised EA baselines, proving its potential for scalable and effective EA across large KGs. |
| Researcher Affiliation | Collaboration | Xiongnan Jin1, Zhilin Wang2, Jinpeng Chen3,4, Liu Yang5, Byungkook Oh6, Seung-won Hwang7, Jianqiang Li1* 1National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University 2Alibaba Group 3School of Computer Science, Beijing University of Posts and Telecommunications 4Xiangjiang Laboratory 5School of Computer Science and Engineering, Central South University 6Computer Science & Engineering, Konkuk University 7Department of Computer Science and Engineering, Seoul National University |
| Pseudocode | No | The paper describes the methodology in the 'Proposed Approach' section, detailing steps like target entity selection, LLM annotation, majority voting, and SLM self-training, but it does so using descriptive paragraph text and a diagram (Figure 1), without presenting any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code and data are available at https://github.com/xnjin-ai/HLMEA. |
| Open Datasets | Yes | We employ nine widely used EA datasets DBP15KZH EN,JA EN,F R EN (Sun, Hu, and Li 2017), DBP15KDE EN,F R EN, DW15K V1, DY15K V1, and DBP100KDE EN,F R EN (Sun et al. 2020). |
| Dataset Splits | Yes | To reduce the utilization of LLMs, 20% of data was employed to train the SLMs with the self-supervision of LLM annotation, then the HLMEA inference results on the remaining 80% were reported. |
| Hardware Specification | Yes | Experiments were conducted on a server running Ubuntu 22.04, which has an AMD Ryzen 9 7950X Processor, 128GB memory, and an NVIDIA RTX A6000 GPU. |
| Software Dependencies | Yes | We implemented HLMEA using Python (version 3.9.18) with a Py Torch (version 2.1.2) backend. |
| Experiment Setup | Yes | LLMs adopted were large-scale close-sourced Chat GPT (Open AI 2022) (version gpt-3.5-turbo-1106) and ERNIE (Baidu Research 2023) (version ERNIE-3.5-8K-0329), and relatively small and open-sourced Qwen (Bai et al. 2023) (version 7B). Pre-trained multi-lingual language models E5 (Wang et al. 2024a), La BSE (Feng et al. 2022), MPNet and Mini LM (Reimers and Gurevych 2019) were employed as SLMs. Adam W (Loshchilov and Hutter 2019) was adopted as the optimizer, and the learning rate was set as 1e 5. LLM annotation repetition n and candidate entity quantity top-k were set in the range [3, 20]. |