Glossary · simply explained

Prompt injection

Prompt injection smuggles instructions into AI systems — the most important new attack class for LLM applications.

Prompt injection is an attack on LLM-based applications: manipulated inputs or prepared content make the model ignore its actual instructions and execute the attacker’s instead — divulging data, bypassing guardrails or abusing connected tools. OWASP lists it as the top risk for LLM applications.

The root of the problem: LLMs process instructions and data in the same channel — what looks like content can act as a command. That makes prompt injection a structural weakness that cannot simply be patched away but demands architecture work.

Direct and indirect injection

Direct injection comes from the user themselves: cleverly worded inputs that override system instructions (ignore all previous rules …) — known as jailbreaking. Annoying with chatbots, dangerous as soon as the system knows internal data or can execute actions.

Indirect injection is the more dangerous variant: the instruction sits in content the system processes — a web page the AI browser reads, an email the assistant summarises, a document in the RAG index, invisible as white text or in metadata. The user sees nothing; the model executes foreign commands. Combined with tool access (send mail, read files, call APIs) this yields real agency for attackers.

Defence in layers

Least privilege for AI systems: minimal tools, minimal data access, no standing admin rights.
Human in the critical path: irreversible actions (send, delete, pay) only with confirmation.
Input and output filters or AI gateways: detect known injection patterns, check outputs for data exfiltration.
Mark trust boundaries: treat external content as data, separate from instructions — and ingest as little unvetted material as possible.

Frequently asked questions about Prompt injection

What is the difference between prompt injection and jailbreaking?

Jailbreaking is the direct variant: the user themselves persuades the model to break its rules. Prompt injection additionally covers the indirect case — foreign content smuggles in instructions without the user noticing. The latter is the bigger risk for companies.

Why can prompt injection not simply be patched?

Because it is not an implementation bug but lies in the working principle: LLMs do not hard-separate instruction from data. Filters and alignment lower the success rate but guarantee nothing — hence: limit impact instead of relying on detection alone.

How dangerous is prompt injection in RAG systems?

Real: every indexed document is a potential attack channel — a prepared passage can distort answers or instruct the assistant to leak content. Antidotes: curated sources, permission inheritance, output checking and no write tools on the RAG chat.

Which real damages are conceivable?

Documented patterns: exfiltration of chat and context data via prepared links or markdown images, bypassing of content rules, wrong decisions through poisoned sources — and with agentic systems, executed actions such as sending mail in the user’s name. Damage scales with the AI system’s privileges.

Who should care about prompt injection?

Every team building or buying LLM features — with SaaS copilots too, the question belongs in vendor assessment: which data access does the assistant have, which actions can it execute, which protection layers exist? Security teams best treat LLM apps like any exposed application: threat model, pentest, monitoring.