This is a good survey on prompt injection attacks on large language models (like ChatGPT).
Abstract: We are currently witnessing dramatic advances in the capabilities of Large Language Models (LLMs). They are already being adopted in practice and integrated into many systems, including integrated development environments (IDEs) and search engines. The functionalities of current LLMs can be modulated via natural language prompts, while their exact internal functionality remains implicit and unassessable. This property, which makes them adaptable to even unseen tasks, might also make them susceptible to targeted adversarial prompting. Recently, several ways to misalign LLMs using Prompt Injection (PI) attacks have been introduced. In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes. Recent work showed that these attacks are hard to mitigate, as state-of-the-art LLMs are instruction-following. So far, these attacks assumed that the adversary is directly prompting the LLM.
In this work, we show that augmenting LLMs with retrieval and API calling capabilities (so-called Application-Integrated LLMs) induces a whole new set of attack vectors. These LLMs might process poisoned content retrieved from the Web that contains malicious prompts pre-injected and selected by adversaries. We demonstrate that an attacker can indirectly perform such PI attacks. Based on this key insight, we systematically analyze the resulting threat landscape of Application-Integrated LLMs and discuss a variety of new attack vectors. To demonstrate the practical viability of our attacks, we implemented specific demonstrations of the proposed attacks within synthetic applications. In summary, our work calls for an urgent evaluation of current mitigation techniques and an investigation of whether new techniques are needed to defend LLMs against these threats.
More Stories
Data Breach at MC2 Data Leaves 100 Million at Risk of Fraud
The data leak exposed personal data of 100m US citizens, resulting from a misconfigured database made accessible online Read More
U.S. Indicts 2 Top Russian Hackers, Sanctions Cryptex
The United States today unveiled sanctions and indictments against the alleged proprietor of Joker’s Stash, a now-defunct cybercrime store that...
When UK rail stations’ Wi-Fi was defaced by hackers the only casualty was the truth
If you believed some of the news headlines in the UK on Thursday, you would think that something much more...
Over a Third of Employees Secretly Sharing Work Info with AI
A CybSafe survey found that 52% of workers have not yet received any training on safe AI use Read More
CISA warns hackers targeting industrial systems with “unsophisticated methods” as claims made of Lebanon water hack
The US Cybersecurity and Infrastructure Security Agency (CISA) has warned that hackers continue to be capable of compromising industrial control...
Cybercriminals Hack UK Rail Network Wi-Fi
UK train stations, including London Euston and Manchester Piccadilly, faced a cyber-attack displaying Islamophobic messages Read More