Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
Authors: Tianle Xia, Liang Ding, Guojia Wan, Yibing Zhan, Bo Du, Dacheng Tao
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments across widely used datasets demonstrate that LACT has substantial improvements (brings an average +5.5% MRR score) over advanced methods, achieving the new state-of-the-art. |
| Researcher Affiliation | Collaboration | Tianle Xia1, Liang Ding2, Guojia Wan1*, Yibing Zhan3, Bo Du1*, Dacheng Tao4 1School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China 2 The University of Sydney 3 JD Explore Academy 4College of Computing and Data Science at Nanyang Technological University, Singapore 639798 EMAIL,EMAIL,EMAIL |
| Pseudocode | No | The Binary Tree Decomposition Mechanism is divided into the following three steps: Query Computation Tree. Existing embedding-based methods rely on DNF to avoid embedding divergence (Ren and Leskovec 2020). Our method bypasses logical operator learning. Consequently, this allows us to convert any query into a logic-equivalent tree form through iterative sub-query recursion. For a complex EFO1 query, like the example shown in Figure 1, its computation graph that is a directed acyclic graph can be converted into a tree where the root node is v?. The tail entities and intermediate entities in complex queries are in one-to-one correspondence with the root nodes and leaf nodes in the generated calculation tree respectively. In view of the reverse order that the tail entity brings to the root node, the edges in the tree corresponding to the relationship in each query are child-to-parent directions. In the Appendix, We demonstrate a simple but systematic method for converting complex queries into computational trees. Binary Tree Decomposition. We split each 1-to-n intersection/union node into n corresponding child nodes. Consider that merging union branches may result in a 1-ton structure consisting of intersection and union edges, as shown in Figure 1. This can be properly addressed by separating v? into an intersection node structure (v 3 and v 5 in Figure 1) firstly, then decomposing the node that was decomposed in the previous layer into a deeper intersection node structure (v 1 and v 2 for v 3, v 4, v3 for v 5), where v denotes an intermediate entity retrieved by Neighborhood Retrieval Algorithm (Seen in Appendix). Reverse Level Traversal. Finally, we decompose the binary computation tree into independent branches. Since the root node of the calculation tree is the answer entity, we perform a hierarchical traversal of all non-leaf nodes of the binary tree in reverse. As shown in Figure 1, the complex EFO1 query is decomposed into a sequence: [(v1, r, v 1), (v2, r, v 2), (v3, r, v 5), (v4, r, v 4), (v 1, r, v 3), , (v 2, r, v 3), , (v 4, r, v 5), , (v3, r, v 5), (v 3, r, v?), , (v 5, r, v?)]. This descriptive text outlines steps but does not present them in a structured pseudocode or algorithm block. |
| Open Source Code | Yes | Code https://github.com/Tianle Xia0914/LACT |
| Open Datasets | Yes | We opt for the most popular datasets: FB15K, FB15K-237, NELL995. We used the training set of the above dataset as the original training data. Detailed information about the dataset and training set is listed in the Appendix. |
| Dataset Splits | Yes | We opt for the most popular datasets: FB15K, FB15K-237, NELL995. We used the training set of the above dataset as the original training data. Detailed information about the dataset and training set is listed in the Appendix. (...) Following previous works (Ren, Hu, and Leskovec 2019), we choose the Mean Reciprocal Rank (MRR) with standard evaluation protocol for evaluating complex reasoning on KG. The detailed setting can be found in the Appendix. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments in the main text. It only mentions model sizes (e.g., "7B models", "13B") which refers to the number of parameters in the language model, not the hardware. |
| Software Dependencies | No | The paper mentions the use of Large Language Models (LLMs) such as Llama/Llama2 but does not specify any version numbers for key software components like deep learning frameworks (e.g., PyTorch, TensorFlow), programming languages, or other libraries used for implementation. |
| Experiment Setup | No | The paper describes a curriculum learning strategy for training (e.g., using 80% easy, 10% medium, and 10% difficult samples for the first stage). However, it lacks specific experimental setup details such as learning rates, batch sizes, number of epochs, optimizer types, or other concrete hyperparameters in the main text, deferring "Detailed information about the dataset and training set" and "detailed setting" to the Appendix. |