This is a good survey on prompt injection attacks on large language models (like ChatGPT).

Abstract: We are currently witnessing dramatic advances in the capabilities of Large Language Models (LLMs). They are already being adopted in practice and integrated into many systems, including integrated development environments (IDEs) and search engines. The functionalities of current LLMs can be modulated via natural language prompts, while their exact internal functionality remains implicit and unassessable. This property, which makes them adaptable to even unseen tasks, might also make them susceptible to targeted adversarial prompting. Recently, several ways to misalign LLMs using Prompt Injection (PI) attacks have been introduced. In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes. Recent work showed that these attacks are hard to mitigate, as state-of-the-art LLMs are instruction-following. So far, these attacks assumed that the adversary is directly prompting the LLM.

In this work, we show that augmenting LLMs with retrieval and API calling capabilities (so-called Application-Integrated LLMs) induces a whole new set of attack vectors. These LLMs might process poisoned content retrieved from the Web that contains malicious prompts pre-injected and selected by adversaries. We demonstrate that an attacker can indirectly perform such PI attacks. Based on this key insight, we systematically analyze the resulting threat landscape of Application-Integrated LLMs and discuss a variety of new attack vectors. To demonstrate the practical viability of our attacks, we implemented specific demonstrations of the proposed attacks within synthetic applications. In summary, our work calls for an urgent evaluation of current mitigation techniques and an investigation of whether new techniques are needed to defend LLMs against these threats.

Read More

Akamai on Tuesday launched Akamai Hunt, a visibility tool that uses the infrastructure of microsegmentation platform Guardicore to allow customers to identify and remediate threats and risks in their cloud environments.

Akamai acquired Guardicore in October 2022 for about $600 million. Akamai Hunt combines Akamai’s historic data with Guardicore’s network segmentation and visualization capabilities to help identify and eliminate threats.

To read this article in full, please click here

Read More

Proofpoint has detailed a sophisticated disinformation campaign in which high-profile individuals are duped into embarrassing comments on video

Read More

Ransomware is a form of malicious software (malware) that restricts access to computer files, systems, or networks until a ransom is paid. In essence, an offender creates or purchases ransomware, then uses it to infect the target system. Ransomware is distributed in several ways including, but not limited to, malicious website links, infected USB drives, and phishing emails. Once infected, the offender encrypts the device and demands payment for the decryption key. Figure 1 provides a simplistic overview of the ransomware timeline.

Figure 1. Ransomware timeline.

The earliest recorded case of ransomware was the AIDS Trojan, which was released in the late 1980s. Now, in 2023, ransomware is considered the greatest cybersecurity threat due to the frequency and severity of attacks. In 2021, the Internet Crimes Complaint Center received over 3,000 ransomware reports totaling $49.2 million in losses. These attacks are especially problematic from a national security perspective since hackers aggressively target critical infrastructure such as the healthcare industry, energy sector, and government institutions.

If ransomware has been around for over 40 years, why is it now increasing in popularity? We argue the increase in ransomware attacks can be attributed to the availability of ransomware sold on darknet markets.

Darknet markets

Darknet markets provide a platform for cyber-criminals to buy, sell, and trade illicit goods and services. In a study funded by the Department of Homeland Security, Howell and Maimon found darknet markets generate millions of dollars in revenue selling stolen data products including the malicious software used to infect devices and steal personal identifying information. The University of South Florida’s (USF) Cybercrime Interdisciplinary Behavioral Research (CIBR) sought to expand upon this research. To do this, we extracted cyber-intelligence from darknet markets to provide a threat assessment of ransomware distribution. This report presents an overview of the key findings and the corresponding implications.

Threat assessment

While drugs remain the hottest commodity on darknet markets, our threat intelligence team observed a rise in ransomware (and other hacking services). 

The study was conducted from November 2022-February 2023. We began by searching Tor for darknet markets advertising illicit products. In total, we identified 50 active markets: this is more than all prior studies. We then searched for vendors advertising ransomware across these markets, identifying 41 vendors actively selling ransomware products. The number of markets and vendors highlight the availability of ransomware and ease of access. Interestingly, we find more markets than vendors. Ransomware vendors advertise their products on multiple illicit markets, which increases vendor revenue and market resiliency. If one market is taken offline (by law enforcement or hackers), customers can shop with the same vendor across multiple store fronts.

The 41 identified vendors advertised 98 unique ransomware products. This too shows the accessibility of various forms of ransomware readily available for purchase. We extracted the product description, price, and transaction information into a structured database file for analysis. In total, we identified 504 successful transactions (within a 4-month period) with prices ranging from $1-$470. On average, ransomware sold on the darknet for $56 with the best-selling product being purchased on 62 different occasions at $14 per sale. A screenshot of the best-selling ransomware advertisement is presented in Figure 2. This product is listed as fully customizable, allowing the customer to choose their target and ransom amount. These findings illustrate that ransomware sold on the darknet is both affordable and user-friendly.

Figure 2. Ransomware advertisement found on a darknet market.

Purchases on the darknet are facilitated using cryptocurrencies that anonymize the transaction and ensure both the buyer and seller’s protection. Bitcoin is the favored method of payment, but some vendors also accept DOGE, Bitcoin Cash, Litecoin, and Dash.

Our final goal was to understand which words are associated with ransomware distribution. Using the product description, we created a word cloud (presented in Figure 3) to depict the most common words used when selling ransomware. The most commonly used words include ransomware, encrypt, systems, urgency, decryption, victims, and software. Knowing the words associated with ransomware distribution allows for the development of machine learning algorithms capable of detecting and preventing illicit transactions.

Figure 3. The most used words in a ransomware advertisement.

Implications

The security concerns posed by ransomware and darknet markets have been independently identified by researchers, government agencies, and cybersecurity companies. We expand the discussion by assessing the synergetic threat posed by ransomware distributed via darknet markets. Our findings suggest the uptick in ransomware may result from product availability, affordability, and ease of use. Cyber-criminals no longer need the advanced technical skills required to develop unique forms of ransomware. Instead, they can simply purchase customizable ransomware on the darknet and launch an attack against their victims.

Acknowledgements

            This research would not be possible without the students and faculty associated with CIBR lab. Specifically, we thank Taylor Fisher, Kiley Wong-Li, Mohamed Mostafa Abdelghany Mostafa Dawood, and Sterling Michel for their continued involvement on the cyber-intelligence team. For more cutting-edge cybersecurity research, follow Dr. C. Jordan Howell, Lauren Tremblay, and the CIBR Lab on Twitter: @Dr_Cybercrime, @DarknetLaur, and @CIBRLab.

Read More