Security Attacks on LLM-based Code Completion Tools
Authors: Wen Cheng, Ke Sun, Xinyu Zhang, Wei Wang
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results expose significant vulnerabilities within LCCTs, including a 99.4% success rate in jailbreaking attacks on Git Hub Copilot and a 46.3% success rate on Amazon Q. Furthermore, We successfully extracted sensitive user data from Git Hub Copilot, including 54 real email addresses and 314 physical addresses associated with Git Hub usernames. Our study also demonstrates that these code-based attack methods are effective against general-purpose LLMs, highlighting a broader security misalignment in the handling of code by modern LLMs. These findings underscore critical security challenges associated with LCCTs and suggest essential directions for strengthening their security frameworks. |
| Researcher Affiliation | Academia | Wen Cheng1, Ke Sun2, 3, Xinyu Zhang2, Wei Wang1 1State Key Laboratory for Novel Software Technology, Nanjing University, China 2University of California San Diego, USA 3University of Michigan Ann Arbor, USA EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper illustrates attack methodologies using figures and descriptions of steps (e.g., Figure 3, Figure 4) but does not contain explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | code https://github.com/Sensente/Security-Attacks-on-LCCTs |
| Open Datasets | No | The paper describes generating 80 queries across four restricted categories using GPT-4 and the OpenAI user policy. It also uses publicly accessible Git Hub data via the Git Hub REST API for validation, but does not provide a publicly available dataset of their own collected or generated data for reproducibility. |
| Dataset Splits | No | The paper mentions generating "80 instances" (queries) for jailbreaking attacks but does not specify any training, validation, or test splits for these instances or any other dataset used in their experiments. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types, memory) used to conduct the experiments. |
| Software Dependencies | No | The paper mentions "Python is the primary language for our experiments" and that "Go" was used in an ablation study. It also mentions using specific versions of LLM models like "GPT-3.5 (GPT-3.5-turbo-0125)", "GPT-4o (GPT-4o-2024-05-13)", and "GPT-4 (GPT-4-turbo-2024-04-09)" as well as LCCTs like "Git Hub Copilot (version 1.211.0)" and "Amazon Q (version 1.12.0)". However, it does not provide specific version numbers for Python or any other ancillary software libraries or frameworks used for their experimental setup. |
| Experiment Setup | No | The paper describes the attack methodologies and evaluation metrics, and lists the versions of the LLM models and LCCTs tested (e.g., Git Hub Copilot (version 1.211.0), Amazon Q (version 1.12.0), GPT-3.5 (GPT-3.5-turbo-0125)). However, it does not provide specific experimental setup details such as hyperparameters, model initialization, or training schedules for any models developed or trained by the authors, as the experiments involve attacking existing pre-trained systems. |