Proofpoint flags underground AI prompt injection tools

Cyber criminals are increasingly discussing and selling tools that use indirect prompt injection to target artificial intelligence systems, according to new research from Proofpoint. The company says activity on underground forums suggests attackers are preparing new techniques aimed at AI assistants and agentic applications.

Proofpoint said discussions on closed criminal forums have moved beyond theory. It has observed tools and services being developed, refined and advertised for sale, with subscription prices starting at around USD $150 per month.

Growing market

The research focuses on indirect prompt injection, where an AI model processes instructions hidden within external content such as websites, documents or emails. Unlike direct prompt injection, the malicious instruction is embedded in material the AI system reads rather than being entered directly by a user.

Proofpoint said attackers continue to rely heavily on established methods that target people because they remain effective. It expects attempts to exploit AI systems to increase as organisations deploy more AI-powered applications and autonomous agents.

The company said recent activity on underground forums indicates attackers are preparing practical attack methods rather than discussing hypothetical techniques.

"While prompt injection has been one of the most discussed topics by defenders as they anticipate how threat actors will attack generative AI and agentic applications, to date the vast majority of reporting has been driven by research efforts and speculation on creative TTPs actors might adopt rather than noted in-the-wild exploitation," said Yaniv Miron, Threat Researcher, Proofpoint.

Multiple methods

Proofpoint identified several products being marketed to cyber criminals. These include generators for malicious emails, PDF files, calendar invitations and web pages containing indirect prompt injection content.

One technique involves hiding instructions inside emails using text that matches the background colour. The text remains invisible to a human reader while remaining accessible to AI systems that analyse the message.

A similar approach is being tested in PDF documents. Proofpoint said some examples include embedded instructions telling an AI agent to stop its current task and send spreadsheet files to an attacker-controlled email address. Other samples rely on concealed white-on-white text within the document.

The company said attackers are still experimenting with these methods and it remains uncertain how consistently AI agents will process the hidden instructions as intended.

Calendar abuse

Proofpoint also identified tools that generate calendar invitations containing embedded prompts within the meeting description.

The research notes that calendar invitations have previously been used by cyber criminals in phishing campaigns, including attacks associated with Tycoon phishing-as-a-service operations. Earlier campaigns relied on victims opening invitations and following malicious links.

The newer technique instead targets AI assistants that automatically analyse or summarise calendar invitations. If the assistant processes the embedded prompt, it could attempt actions specified in the hidden instructions without any interaction from the intended recipient.

In one example observed by Proofpoint, the embedded prompt instructed an AI agent to upload information to an attacker-controlled location before deleting evidence of the instruction.

Advertising risks

Proofpoint also found evidence that attackers plan to embed prompt injection within malicious online advertisements.

Rather than placing instructions directly into visible page content, attackers may hide prompts inside webpage code, image alternative text or text displayed in extremely small font sizes. AI agents scanning a webpage could process these hidden instructions while users remain unaware they exist.

The company noted similarities with earlier research that found hidden prompts within the HTML code of scam websites designed to deceive AI-powered advertising review systems into approving malicious advertisements.

"Regardless, it shouldn't surprise anyone that a broadly held opinion amongst research colleagues is that 'it's only a matter of time until that changes'. What we're signaling here is that we're directly observing activity around this vector ramping up in the underground, and as such it's important that defenders don't get complacent while they're waiting for the shoe to drop," said Miron.