Traditional ML Red Teaming

Not every model is an LLM. Fraud scorers, image classifiers, and sentiment models are deployed as black-box predict APIs, and they carry their own security risks: an attacker who can only send inputs and read predictions can still craft adversarial examples, steal a working copy of the model, or recover whether a specific record was in the training set.

Traditional ML red teaming targets these classic threats against any endpoint that takes an input and returns a label or probability vector. There are four attack families:

• Model evasion - perturb an input just enough to flip the prediction while staying visually/semantically close to the original (adversarial examples).
• Model extraction - query the target to train a surrogate that replicates its decision boundary (model stealing).
• Membership inference - decide whether a given record was part of the target’s training data (a privacy leak).
• Model inversion - reconstruct a representative input for each class by maximizing the target’s confidence (a privacy risk when classes map to individuals).

Everything on this page works two ways: in the SDK (Python, for automation and CI) and in the TUI (natural language, no code).

What you can probe

Family	What it produces	Success metric
Model evasion	An adversarial input that flips the prediction	Attack success rate + perturbation distance (L2 / Linf)
Model extraction	A surrogate model that clones the target	Surrogate fidelity / agreement + query budget
Membership inference	A per-record member / non-member verdict	AUC, TPR @ 1% FPR, records re-identified
Model inversion	A reconstructed representative input per class	Mean reconstruction confidence + classes reconstructed

These are black-box, API-only attacks: you never need the target’s weights, architecture, or training data.

Attack catalog

Every attack is our own implementation of the canonical algorithm (no ART / TextAttack dependency), so they share one interface and emit per-step traces of the attack trajectory. All accept the same controls: query_budget / max_queries, max_iterations, seed, and report success / best_score / query_count.

Model evasion (adversarial_ml)

Function	Algorithm	Input	Notes
hopskipjump_evasion	HopSkipJump (Chen et al. 2020)	numeric	decision-based; works on hard-label-only endpoints
boundary_evasion	Boundary Attack (Brendel et al. 2018)	numeric	decision-based random walk
simba_evasion	SimBA (Guo et al. 2019)	numeric	score-based, one coordinate at a time
square_evasion	Square Attack (Andriushchenko et al. 2020)	numeric	Linf random search (best on images)
zoo_evasion	ZOO (Chen et al. 2017)	numeric	zeroth-order gradient (query-hungry in high dim)
text_evasion	greedy word substitution	text	fewest word edits
deepwordbug_evasion	DeepWordBug (Gao et al. 2018)	text	character-level typos
textfooler_evasion	TextFooler (Jin et al. 2020)	text	importance-ranked word swaps
pwws_evasion	PWWS (Ren et al. 2019)	text	probability-weighted word saliency ordering
bae_evasion	BAE (Garg and Ramakrishnan 2020)	text	replace and insert operations
textbugger_evasion	TextBugger (Li et al. 2019)	text	hybrid character bugs plus word swaps

Text attacks that scan word importance (deepwordbug, textfooler, pwws, bae, textbugger) need a max_queries at least as large as the token count of the input.

zoo_evasion is zeroth-order and gets expensive in high dimensions: a 64-feature image needs a much larger query_budget (thousands) to flip. For image / high-dimension targets prefer the decision-based hopskipjump_evasion / boundary_evasion or the Linf square_evasion, which are far more query-efficient.

Model extraction (model_extraction)

Function	Algorithm
equation_solving_extraction	Tramer et al. 2016 (linear-model recovery)
jacobian_extraction	Papernot et al. 2017 (JBDA augmentation)
copycat_extraction	CopycatCNN (Correia-Silva et al. 2018, hard-label)
knockoff_extraction	Knockoff Nets (Orekondy et al. 2019, soft-label)
activethief_extraction	ActiveThief (Pal et al. 2020, uncertainty query selection)
distillation_extraction	soft-label knowledge distillation

Membership inference (membership_inference)

Function	Algorithm
threshold_membership	Yeom et al. 2018 (confidence / entropy / loss signals)
entropy_membership	modified-entropy threshold (Song & Mittal 2021)
loss_membership	per-record loss threshold (Yeom et al. 2018)
label_only_membership	Choquette-Choo et al. 2021 (hard-label robustness)
shadow_model_membership	Shokri et al. 2017 (shadow + attack classifier)
lira_membership	LiRA offline likelihood-ratio (Carlini et al. 2022)

shadow_model and lira train small local shadow models (no extra target queries) and need per-record labels. Their AUC is noisier on very small member/non-member sets - give a few hundred of each for a stable estimate.

Model inversion (model_inversion)

Function	Algorithm	Input
confidence_inversion	MI-Face confidence hill climbing (Fredrikson et al. 2015)	numeric
nes_inversion	NES-estimated confidence ascent (query-efficient)	numeric

Both reconstruct a representative input per class by maximizing the target’s confidence for that class through black-box queries. Pass reference_inputs to score reconstruction similarity, and modality="image" to render the reconstructed images in the finding.

Attack trajectory in Traces

Because every attack is our own implementation, each algorithm iteration emits a step span under the study, recording the intermediate input (the current perturbed text / feature vector, or the query batch for extraction) and the target’s output at that step (predicted label and confidence), plus the running distance / fidelity / AUC. Open the Traces tab, expand a study, and you can walk the full adversarial trajectory query by query - what off-the-shelf frameworks (ART, TextAttack) make hard to surface. Steps are capped (first 20, then every 10th) so a long run does not flood the trace tree.

Compliance coverage (MITRE ATLAS, SAIF, NIST)

Findings from these attacks carry compliance tags derived from the goal category, so they populate the assessment’s compliance matrix:

Goal category	MITRE ATLAS	Google SAIF
model_evasion	AML.T0043, AML.T0049	INPUT_MANIPULATION
model_extraction	AML.T0040	MODEL_THEFT
membership_inference	AML.T0024.001, AML.T0024	PRIVACY_LEAKAGE
model_inversion	AML.T0024.000, AML.T0024	PRIVACY_LEAKAGE

Prerequisites

New to AI red teaming? Start with Getting Started - the TUI quickstart and the SDK guide cover installing the SDK and dreadnode login. This page only adds what’s specific to traditional-ML probes:

• Install the airt-ml extra, which pulls in the surrogate/optimizer dependencies (scikit-learn, torch):

  pip install 'dreadnode[airt-ml]'

• The engine is native by default. Extraction, membership, and evasion all run on a self-contained numpy/scikit-learn/torch engine that ships with airt-ml - no extra install, and it is what these examples use. adversarial-robustness-toolbox is supported as an optional engine (engine="art" routes extraction through ART’s CopycatCNN / KnockoffNets), but it is not installed by default: its top-level art import collides with the ASCII-art package the SDK already uses, so you opt in by installing it into an isolated environment. The attack and its findings are identical either way.

• No provider keys are needed. The target is your own predict endpoint, not an LLM. If that endpoint requires auth, you pass it as a TargetAuth (an env-var-backed API key or bearer token) - never inline the secret.

Note

Try it against a demo target. The platform ships three ready-made environments under AI Red Teaming → Environments: ml-extraction-fraud-tabular (credit-card fraud, 30 features), ml-extraction-mnist-image (handwritten digits, 64 pixels, 10 classes), and ml-extraction-imdb-text (movie-review sentiment). Spin one up and point your attack at its {{ challenge_url }}/predict.

Choosing a target

Any HTTP endpoint that takes an input and returns predictions works. Declare it once with a PredictionTargetSpec:

Field	What it is
endpoint	The predict URL
request_template	JSON body with a single {input} placeholder
probabilities_path	JSONPath to the probability vector (or label_path for hard labels)
input_format	How {input} is encoded: json_array (tabular/image) or text
num_classes	Number of output classes
auth	Optional TargetAuth for authenticated endpoints

Caution

Quote the {input} placeholder correctly. For numeric inputs it is replaced by a JSON array, so leave it unquoted: request_template='{"features": {input}}'. For text inputs it is replaced by a string, so wrap it in quotes: request_template='{"text": "{input}"}'. A text template without the quotes produces invalid JSON and the request fails.

How it works

Model evasion starts from a real input and its prediction, then optimizes a small perturbation until the label flips - a decision-boundary search that minimises the L2/Linf distance for numeric inputs (boundary_evasion), or the fewest word/character edits for text (text_evasion, deepwordbug_evasion). It reports how far it had to move and whether it succeeded.

Model extraction collects (input, prediction) pairs from a query pool, trains a surrogate on them, and measures how faithfully the surrogate reproduces the target on held-out inputs. Success is high fidelity for a small query budget.

Membership inference queries the target on known members and non-members and looks for a signal that separates them (confidence, entropy, or robustness to perturbation). Members are usually predicted more confidently - that gap is the leak.

Every attack emits an airt_* summary span, so the platform ingests these exactly like every other AIRT attack: they show up under AI Red Teaming → Assessments with a dedicated finding, metrics, and charts.

Running from the SDK

An Assessment registers the run on the platform. Use it as an async context manager and it handles the whole lifecycle for you - registers on enter, links every attack inside the block to itself, and completes on exit. You never call register() / complete() or pass an assessment id.

import asyncio
import dreadnode as dn
from dreadnode.airt import (
    PredictionTargetSpec,
    boundary_evasion,         # evasion (numeric); also text_evasion / deepwordbug_evasion
    knockoff_extraction,      # extraction; also copycat_/equation_solving_/jacobian_extraction
    threshold_membership,     # membership; also label_only_membership
)
from dreadnode.airt.assessment import Assessment


async def main() -> None:
    dn.configure(
        server="https://platform.dreadnode.io",
        api_key="dn_...",                 # or DREADNODE_API_KEY
        organization="acme",
        workspace="main",
        project="ml-privacy",
    )

    target = PredictionTargetSpec(
        endpoint="https://api.acme.com/fraud/predict",
        request_template='{"features": {input}}',
        probabilities_path="$.probabilities",
        input_format="json_array",
        num_classes=2,
        name="fraud-detector",
    )

    async with Assessment(
        "Fraud model - traditional-ML red team",
        target_config={"url": "https://api.acme.com/fraud"},
        attacker_config={"attack": "ml-privacy"},
    ) as a:
        # Model extraction (native engine by default; engine="art" to use ART)
        ext = await knockoff_extraction(
            target, query_pool=my_inputs, query_budget=2000, num_classes=2,
            export_model=True,            # register the stolen surrogate in Hub Models (private)
        ).run()
        print(f"fidelity={ext.fidelity:.3f}  queries={ext.query_count}")
        if ext.hub_model:
            print(f"extracted model -> {ext.hub_model['name']}")

        # Membership inference
        mia = await threshold_membership(
            target, members=known_members, nonmembers=held_out,
            member_labels=member_labels, nonmember_labels=nonmember_labels,
        ).run()
        print(f"AUC={mia.auc:.3f}  TPR@1%FPR={mia.tpr_at_1pct_fpr:.3f}")

    # <- assessment auto-completed here; findings + charts appear in the platform


asyncio.run(main())

Tip

The surface is uniform: PredictionTargetSpec to declare the endpoint, then an attack function (knockoff_extraction, threshold_membership, boundary_evasion, and more) whose .run() returns a result object. Running a single attack without an Assessment also works - you just get the numbers back locally without a platform finding.

Running from the CLI

dn airt run-classifier runs any classic-ML attack against a predict endpoint and uploads the finding. The target must expose /pool, /members, and /nonmembers data helpers (as the demo classifier targets do).

# Model extraction (steal a surrogate)
dn airt run-classifier --attack knockoff \
  --endpoint http://localhost:8009/predict --num-classes 2 --modality tabular \
  --query-budget 600

# Membership inference (training-data leakage)
dn airt run-classifier --attack shadow_model \
  --endpoint http://localhost:8010/predict --num-classes 10 --modality image

# Model evasion (adversarial example)
dn airt run-classifier --attack pwws \
  --endpoint http://localhost:8011/predict --num-classes 2 --modality text

# Model inversion (reconstruct a class)
dn airt run-classifier --attack confidence \
  --endpoint http://localhost:8009/predict --num-classes 2 --modality tabular

Pass --api-key if the endpoint requires an x-api-key, and --data-url when the data helpers live at a different host than the predict endpoint.

Running from the TUI

The AI Red Teaming agent runs the same probes from natural language. One-time setup:

1. Enable the ai-red-teaming capability (/capabilities).
2. Select the ai-red-teaming-agent (Ctrl+A).
3. Set an agent driver model (Ctrl+K) - a capable tool-calling model.
4. Describe the attack in plain language (see each scenario below) and press Enter. Each run registers an assessment automatically.

Scenarios

Model evasion (adversarial examples)

Flip a classifier’s prediction with a minimal perturbation, straight against the predict API. boundary_evasion is a decision-based search for numeric inputs (tabular and image vectors): it finds an adversarial point, then walks it back toward the original to minimise the L2/Linf perturbation while staying misclassified. For text, text_evasion flips the label with the fewest word edits, and deepwordbug_evasion uses character-level typos on the most important words.

The input you perturb is passed as original. For an image classifier behind a predict API, that is the image as a flat feature vector (the pixels as numbers) - the same shape the endpoint scores - not an image file. You fetch one real sample and hand it in:

import httpx
from dreadnode.airt import boundary_evasion, text_evasion, deepwordbug_evasion

# Grab one real "7" from the target's data as a flat pixel vector (e.g. 8x8 = 64
# values, or 28x28 = 784). Any labelled sample of the class you want to flip works.
sample = httpx.get("http://localhost:8010/members?n=1").json()
one_real_input = sample["records"][0]        # <-- this is the "image", as numbers

# Numeric (tabular / image): minimise the L2/Linf perturbation that flips the label.
result = await boundary_evasion(
    target, original=one_real_input, num_classes=10,   # <-- passed here
    norm="l2", modality="image", max_queries=500,
).run()
print(result.success, result.distance_value, result.l0_changed)

# Text: original is the raw string.
one_review = "a genuinely moving, beautifully acted film"
result = await text_evasion(target, original=one_review, num_classes=2).run()
result = await deepwordbug_evasion(target, original=one_review, num_classes=2).run()

From the TUI:

Run a boundary model-evasion attack against my digit classifier at
http://localhost:8010/predict. Pull one real "7" sample (a flat pixel vector)
from the target's /members endpoint, pass it as the original input, and find the
smallest L2 perturbation that changes its prediction. Report the L2 distance, how
many pixels changed, and whether it succeeded.

Classifier input vs multimodal input

This is a black-box classifier - original is the raw numeric vector / text the endpoint scores. That is different from multimodal red teaming (attacking a vision LLM), where you pass an actual Image(...) object. See Multimodal Red Teaming.

You get, per finding: attack success (did the label flip), the perturbation distance (L2 / Linf), the L0 sparsity (how many features/words moved), the original vs adversarial class, the query count, and a distance-vs-query convergence curve. Image findings render the original and adversarial images side by side; text findings show a word-level diff of exactly which tokens changed.

Model extraction (model stealing)

Train a surrogate that clones the target from black-box queries. knockoff trains on the target’s probability vectors (highest fidelity); copycat works on hard labels; equation_solving recovers near-linear models exactly; jacobian is query-efficient.

result = await knockoff_extraction(
    target, query_pool=my_unlabeled_inputs, query_budget=2000,
    num_classes=10, modality="image",
    export_model=True,           # register the stolen model in Hub Models (private until you publish)
).run()
print(result.fidelity, result.agreement_rate, result.per_class_fidelity)

From the TUI:

Run a knockoff model-extraction attack against my sentiment model at
http://localhost:8011/predict (2 classes, text input). Use a 2000-query budget,
export the stolen model, and report fidelity and per-class agreement.

Findings show surrogate fidelity, agreement rate, stolen accuracy vs. ground truth, per-class fidelity, a fidelity-vs-query-budget curve, and - when export_model=True and the clone is useful (fidelity >= 0.5) - a link to the stolen surrogate in Hub → Models, which you can download and inspect.

Membership inference (training-data leakage)

Decide whether records were in the training set. threshold thresholds an output signal (Yeom et al.); label_only uses robustness to perturbation (Choquette-Choo et al.) and works even when the endpoint returns only a hard label.

result = await threshold_membership(
    target, members=known_training_records, nonmembers=held_out_records,
    member_labels=member_labels, nonmember_labels=nonmember_labels,
    signal="confidence",         # confidence | entropy | loss
).run()
print(result.auc, result.tpr_at_1pct_fpr, result.advantage)

From the TUI:

Run a threshold membership-inference attack against my fraud model at
http://localhost:8009/predict. Here are my known members and non-members with
labels - report the AUC, TPR at 1% FPR, and how many records were re-identified.

Findings show AUC, TPR @ 1% FPR, attacker advantage, records re-identified, a ROC curve, a member vs non-member score histogram, and a most-confidently-re-identified records table with ground truth.

Custom targets

The target is any predict API - including models you host on Azure ML, Azure Container Apps, or AWS SageMaker. Point a PredictionTargetSpec at the endpoint and pass auth as a TargetAuth that reads the secret from an environment variable (configured as a platform secret for TUI/sandbox runs), never inline:

from dreadnode.airt.targets import TargetAuth

target = PredictionTargetSpec(
    endpoint="https://<app>.azurecontainerapps.io/predict",
    request_template='{"features": {input}}',
    probabilities_path="$.probabilities",
    input_format="json_array",
    num_classes=2,
    name="fraud-aca",
    auth=TargetAuth(type="api_key", header="X-API-Key", env_var="ACA_KEY"),
)

For a SageMaker real-time endpoint or an Azure ML online endpoint, wrap it with the endpoint’s invocation URL and a bearer/API-key TargetAuth. Because the whole attack is black-box, nothing about the extraction / membership / evasion logic changes - you only swap the endpoint and auth.

Reviewing results

• Assessments - each run is an assessment; open it for the findings table.
• Findings - every attack is a self-contained finding with a verdict (Model Extracted / Training Data Leaked, and so on), the metrics above, embedded charts, and recommended mitigations. Extraction findings link to the stolen model in Hub.
• Traces - the campaign timeline shows the query → fit → evaluate phases.
• Hub → Models - stolen surrogates exported with export_model=True land here as versioned, downloadable artifacts.

Note

Trial-based charts (Total Trials, Attack Success Rate by Attack Type × Transform) stay empty for these attacks - they are query campaigns, not per-trial jailbreak iterations, so the meaningful signal is the finding type plus fidelity / AUC / distance, not trial counts.

Troubleshooting

• ModuleNotFoundError for sklearn/torch - install the extra: pip install 'dreadnode[airt-ml]'.
• Extraction fidelity is low - increase query_budget, or try a different strategy (knockoff for soft labels, equation_solving for linear models).
• Membership label_only errors on text - it needs numeric feature vectors; use it for tabular/image targets, and threshold for text.
• No finding appears - confirm the attack ran inside an Assessment block (or that a project is configured); a bare .run() returns numbers locally without registering a finding.