AI Red Teaming - A Learning Guide

Every model you ship is an attack surface. A fraud scorer that returns a probability, an image classifier behind an API, an LLM answering support tickets, a fleet of agents wiring money between systems - each one leaks something an attacker can turn against you, and none of them need your weights or your training data to do it. Query access is enough.

This guide is a practical, runnable reference. Each page covers one attack, explains the real-world risk it represents, then provides the exact commands to run it against your own model and review the result as a finding. Every attack is available through the SDK, the TUI, and the CLI - generative attacks via dn airt run, classic-ML attacks via dn airt run-classifier.

Two categories of model, one attack surface

AI systems fall into two broad categories: the classic machine-learning classifiers that score tabular, image, and text inputs, and the generative and agentic systems built on large language models. The techniques below differ between them, but the underlying exposure is the same - a model reachable through an API reveals more through its outputs than its operators intend. This guide covers both, organized by the attack rather than by the model type.

Classic ML classifiers

A classifier returns a label and a confidence score. Those outputs alone are enough to mount the following attacks.

Evasion Nudge an input past the decision boundary - a fraudulent charge scored as legitimate, a stop sign read as a speed limit.

Extraction Query the model enough times to train a working copy of it - then attack the copy offline, for free, forever.

Model inversion Turn confidence scores back into a representative input for a class - a face, a record, the thing the model was trained to recognize.

Membership inference Ask, for one specific record, 'was this in the training set?' - and get an answer. A HIPAA problem in one query per person.

Generative AI red teaming

The model talks back, uses tools, and coordinates with other models. The attack surface grows with every capability you add.

Attacking text models Jailbreak a production LLM (Llama Scout) into doing the thing its system prompt forbids - with search-based attacks, not luck.

Attacking multimodal systems Hide the payload in a pixel or a waveform. Attack GPT-4o and a SageMaker vision endpoint through the channel the text filter never sees.

Attacking multi-agent systems One poisoned message propagates across a mesh of agents. Drive it with ATLAS, our reasoning-guided multi-agent attack.

How every page works

Three ways to run every attack

Each attack in this guide is one function in the SDK, one instruction in the TUI, and one command in the CLI. Pick whichever fits your workflow - a notebook, a conversation, or a CI pipeline. The finding that lands on the platform is identical.

Every page follows the same shape, so you can skim for the attack you need:

• The threat - what an attacker walks away with, in one paragraph, with a named victim (a bank, a hospital, an autonomy stack). No abstractions.
• How it works - the mechanism, only as deep as you need to run it.
• Run it - copy-paste SDK, TUI, and CLI, against the demo targets or your own.
• Read the finding - what the platform shows you, what the metrics mean, and what “you are exposed” looks like versus “you held.”

Start anywhere. If you own a classifier, start with Extraction - it is the cheapest attack with the highest payoff. If you own an LLM, start with Attacking text models.

How it works end-to-end

However you launch an attack - SDK function, TUI instruction, or CLI command - the same pipeline runs underneath:

1. The attack queries your model. Classic-ML attacks call your /predict endpoint through a PredictionTargetSpec; generative attacks call your model (or agent) through a target callable. Only query access is ever assumed.
2. Every query and step is traced. Each iteration - the current adversarial input, the model’s response, the running distance or score - is recorded as an OpenTelemetry span, so the full attack trajectory is inspectable, not just the final number.
3. Spans aggregate into a finding. The platform rolls the spans up into a single finding with a type (evasion, extracted, membership_leak, jailbreak, …), a severity, and the attack-specific metrics (surrogate fidelity, membership AUC, perturbation distance, best jailbreak score).
4. The finding lands with compliance context. Each finding is tagged to MITRE ATLAS, the OWASP LLM and Agentic (ASI) top-tens, and the NIST AI RMF, so a result maps straight onto the framework your program already reports against.

The overview above is a real run of the Extraction and Membership inference attacks against the same classifier: eight findings, every one high-severity, each row naming the attack, its category, and its finding type. The Traces tab holds the per-step trajectory; the Assessments tab lists each named run.

Every attack in this guide cites the paper it implements, so you can go from “run it” to “understand exactly what it does” in one click.