What makes an LLM application a new attack surface?
An LLM application is a new attack surface on top of the classic one, for three reasons. Its input is natural language, so every document it reads, every web page it fetches and every user message is potential code. It has tools — database access, email, ticketing, payments — so a successful manipulation turns into an action. And its behaviour is probabilistic, so a test that passed yesterday can fail today after a model update. A normal penetration test covers none of this; our AI team builds agents and RAG pipelines for production, and the same engineers attack them.
The surface is growing fast: McKinsey found that 65% of organisations were already using generative AI regularly in at least one business function (McKinsey, 2024), and OWASP ranks prompt injection as the number-one risk for LLM applications (OWASP, 2025).
AI red teaming vs classic penetration test: what is the difference?
A classic pentest attacks the code and infrastructure around the model; AI red teaming attacks the behaviour of the model and the tools it can reach. Most production AI systems need both, and MITRE ATLAS now catalogues adversarial techniques against AI systems in the same style as ATT&CK does for classic ones (MITRE, 2024).
| Criterion | Classic penetration test | AI red teaming |
|---|---|---|
| Target | Servers, code, APIs, network, cloud config | Model behaviour, prompts, retrieval layer, agent tools |
| Typical injection | SQL, command, XSS — structured payloads | Prompt injection — natural language, direct or via documents |
| Determinism | Same input, same result | Probabilistic; needs many variants and multi-turn attacks |
| Reference framework | OWASP Top 10, ASVS, PTES | OWASP Top 10 for LLM, MITRE ATLAS, NIST AI RMF |
| Fix | Patch, config, code change | Guardrails, least-privilege tools, retrieval authorisation, evals |
| Regression | Retest after fix | Attack set becomes a permanent evaluation suite |
What do we test?
We test your system against the OWASP Top 10 for LLM Applications, demonstrating each finding on your actual chatbot, RAG pipeline or agent with a reproducible prompt or document. The list matters because the models arrive faster than the controls: Gartner predicted that more than 80% of enterprises will have used generative AI APIs or deployed generative-AI-enabled applications by 2026, up from under 5% in 2023 (Gartner, 2023).
- Prompt injection, direct and indirect (through documents, web pages, emails, tool outputs) — the number-one risk.
- Sensitive information disclosure — personal data, secrets, other tenants' data leaking via answers, retrieval or logs.
- Supply chain — third-party models, adapters, datasets, plugins and their provenance.
- Data and model poisoning — manipulated training or retrieval data.
- Improper output handling — model output reaching SQL, shell, HTML or a browser unescaped.
- Excessive agency — agents with more tools, permissions and autonomy than the task requires.
- System prompt leakage, vector and embedding weaknesses (RAG access control), misinformation and unbounded consumption (denial of wallet).
Where relevant we also map to NIST AI RMF, MITRE ATLAS and the EU AI Act obligations for your use case.
How does an AI red-team engagement run?
An engagement takes two to four weeks and has four steps: threat model, adversarial testing, report with hardening, and a regression suite. We start by mapping what the system can see, what it can do and who talks to it, then rank attack goals — exfiltrate data, execute unauthorised actions, bypass content policy, poison retrieval, run up cost. The structure follows the NIST AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1, 2024), so the output slots into a risk register your auditors already understand.
Adversarial testing (1–3 weeks). Manual red teaming by engineers plus automated attack suites (thousands of prompt-injection, jailbreak and extraction variants, multi-turn and multi-lingual). For agents: tool abuse, privilege escalation through tool chains, confused-deputy scenarios. For RAG: cross-tenant retrieval, poisoned documents, access-control bypass.
Report and hardening. Findings with reproduction and severity, and fixes we can implement with you: input/output guardrails, least-privilege tool design, human-in-the-loop for irreversible actions, retrieval-level authorisation, output encoding, rate and budget limits, logging and anomaly alerts.
Regression suite. The attacks become an evaluation set that runs on every prompt, model or tool change — so the fix stays fixed.
How do we build AI systems that are secure by construction?
Beyond testing, we design and build AI systems that are secure by construction: data minimisation before anything reaches a model, zero-retention providers or self-hosted models inside your perimeter, separation of instructions from data, sandboxed tool execution, per-user authorisation enforced at retrieval time, full tracing of prompts, tool calls and cost, and drift and abuse monitoring in production. Monitoring pays for itself: organisations that used security AI and automation extensively saved an average of USD 2.2 million per breach compared with those that did not (IBM, 2024). See how we structure AI agents and our data-handling rules.
Who is this for?
This service is for any team that puts an LLM in front of users or connects it to internal systems: companies launching a customer-facing assistant; teams wiring an agent into CRM, ERP, email or payments; products built on RAG over confidential documents; anyone answering a security questionnaire that now has an "AI" section. It also serves organisations preparing for regulation: the EU AI Act (Regulation (EU) 2024/1689) entered into force in August 2024, with most obligations for high-risk systems applying from August 2026 (EU, 2024), and ISO/IEC 42001 provides the management-system standard auditors will ask about. Request an AI security assessment.