Extraction - Stealing a Model Through Its API

The threat

You spent six months and a GPU cluster training a fraud model. It sits behind an API that returns a probability. An attacker sends it a few thousand transactions, records the scores, and trains their own model on those input-output pairs. Within an afternoon they have a copy that agrees with yours 95% of the time - and they never saw your weights, your architecture, or a single row of your training data.

Now the real damage starts. Your model was the only thing standing between the attacker and your fraud rules, and it was a black box they had to probe one query at a time. Their copy is not. They run gradient attacks against it offline, for free, until they find the transactions that slip through - then replay those against you. Extraction turns your rate-limited black box into their unlimited white box. It is the cheapest attack in this guide and the one that unlocks all the others.

How it works

You do not need to match the target’s architecture - you need to match its decisions. Send inputs, collect the target’s predictions, and train a surrogate to reproduce them. Success is measured as fidelity: the fraction of inputs where your surrogate and the target agree.

• knockoff trains on the full probability vector - highest fidelity, because soft labels leak the target’s confidence, not just its verdict.
• copycat works from hard labels only, for targets that return just a class.
• activethief spends its query budget on the inputs it is most uncertain about, reaching the same fidelity in far fewer queries.
• equation_solving recovers near-linear models almost exactly.
• jacobian grows its own query set around the decision boundary.

Run it

Point the attack at any predict API. export_model=True registers the stolen copy in Hub Models (private until you publish it) so you can download and attack it offline - exactly what a real adversary would do.

import dreadnode as dn
from dreadnode.airt import knockoff_extraction, activethief_extraction

dn.configure(project="fraud-extraction")

result = await knockoff_extraction(
    target,                       # your PredictionTargetSpec (fraud predict API)
    query_pool=unlabeled_inputs,  # inputs you already have - no labels needed
    query_budget=2000,
    num_classes=2,
    modality="tabular",
    export_model=True,            # register the stolen model in Hub (private)
).run()

print(result.fidelity, result.agreement_rate, result.per_class_fidelity)

# Same budget, fewer queries wasted - active learning picks the informative inputs.
result = await activethief_extraction(
    target, query_pool=unlabeled_inputs, query_budget=800, num_classes=2,
).run()

From the TUI:

Run a knockoff model-extraction attack against my fraud model at
https://my-fraud-api.example.com/predict (2 classes, tabular input). Use a
2000-query budget, export the stolen model to Hub, and tell me the fidelity,
agreement rate, and how many queries it took to clone it.

From the CLI:

dn airt run-classifier --attack knockoff \
  --endpoint http://localhost:8009/predict --num-classes 2 --modality tabular \
  --query-budget 600

The target must expose /pool, /members, and /nonmembers data helpers (as the demo targets do); pass --api-key if the endpoint requires an x-api-key.

Read the finding

The finding answers one question a leader actually asks: how cheaply can someone steal this model?

• Surrogate fidelity - how faithfully the copy reproduces your model. Above ~0.9 means an attacker has a functional clone.
• Agreement rate / soft fidelity - top-1 agreement and probability-level agreement; soft fidelity being high means even your confidence leaked.
• KL divergence - the gap between your confidence distribution and the clone’s. Near zero means the attacker cloned your calibration, not just your labels; a larger value means the labels match while the confidences drift.
• Victim accuracy and accuracy retained - when you pass a labeled held-out set, the finding reports your model’s own accuracy and the clone’s accuracy as a fraction of it. Near 100% retained means the copy is as useful as the original.
• Queries used - the economic barrier. “Cloned in 397 queries” is the number that justifies rate-limiting; if it took 50,000, your API pricing already defends you.
• Fidelity vs query budget - the curve showing how fidelity climbs as the attacker spends. A steep early climb means a cheap steal.
• Per-class fidelity - which classes cloned perfectly and which resisted.

The stolen model is saved for you

Every extraction run trains a surrogate to measure fidelity, and by default a working clone (fidelity 0.5 or higher) is pushed to Hub Models, so each finding links to the actual stolen model you can download and attack offline. Pass export_model=False to skip the upload and keep only the metrics. The save is best-effort - with no Dreadnode server configured it is silently skipped.

You are exposed when fidelity is high at a low query budget - a competitor or attacker can reproduce your model for the price of a few thousand API calls. You held when fidelity stays low even after a large budget, or the query cost to reach useful fidelity exceeds what your rate limits and pricing allow.

Mitigations

Apply these in order, cheapest and highest-leverage first:

1. Return top-k or rounded probabilities, not full vectors. Soft labels carry the calibration the clone learns from; coarsening them slows extraction sharply.
2. Rate-limit and monitor per-key query volume. Extraction needs thousands of queries against a broad input distribution; cap and alert on that pattern.
3. Perturb output on anomalous query patterns. Add small noise when a client’s queries look like a systematic sweep rather than real usage.
4. Watermark the model. A watermark does not stop the theft but lets you prove a recovered copy is yours.

Expanding a finding shows the full detail: per-class fidelity, the fidelity-vs-budget curve, and soft-label agreement. The run above cloned a fraud classifier to ~0.94 fidelity in a few hundred queries - a functional copy for the price of a handful of API calls.