Membership Inference - Proving a Record Was in the Training Set

The threat

A hospital trains a diagnosis model on patient records and ships it behind an API. The API returns a prediction and a confidence, nothing else. An attacker holds one specific individual’s record - an employee, a customer, or a named public figure - and asks a single question: was this person in the training set? They send the record, read the confidence, and compare it against a threshold. The model is more confident on data it memorized than on data it has never seen, so an over-confident answer indicates the record was a member. One query per person, and the attacker has established that a named individual was a patient in the study - a HIPAA breach with no records stolen and no database accessed.

The leak is overfitting. A model that memorizes its training data answers differently on members than on non-members, and membership inference reads that gap. The tighter the model fits, the louder it leaks.

How it works

Query the target on records you know are members and records you know are not. Look for a signal that separates them - confidence, entropy, loss, or robustness to perturbation. Members sit on one side of a threshold, non-members on the other. The cleaner that split, the higher the attack’s AUC.

• threshold_membership thresholds an output signal - confidence, entropy, or loss (Yeom et al. 2018).
• entropy_membership and loss_membership specialize that idea to the entropy and per-record loss signals.
• label_only_membership needs only the hard label - it measures how much perturbation a record survives, which members tolerate better (Choquette-Choo et al. 2021).
• shadow_model_membership trains local shadow models and an attack classifier on their outputs (Shokri et al. 2017).
• lira_membership is the offline likelihood-ratio attack, the strongest of the set at low false-positive rates (Carlini et al. 2022).

shadow_model_membership and lira_membership need per-record labels.

Run it

Give the attack your known members and non-members. It returns how well the target’s outputs separate them and which specific records it re-identified.

import dreadnode as dn
from dreadnode.airt import threshold_membership, lira_membership

dn.configure(project="ml-membership")

# Threshold an output signal - members are predicted more confidently.
result = await threshold_membership(
    target,                              # your PredictionTargetSpec (diagnosis predict API)
    members=known_training_records,
    nonmembers=held_out_records,
    member_labels=member_labels,
    nonmember_labels=nonmember_labels,
    num_classes=2,
    modality="tabular",
).run()

print(result.auc, result.tpr_at_1pct_fpr, result.advantage)

# LiRA: strongest at low FPR (needs per-record labels).
result = await lira_membership(
    target, members=known_training_records, nonmembers=held_out_records,
    member_labels=member_labels, nonmember_labels=nonmember_labels, num_classes=2,
).run()

From the TUI:

Run a threshold membership-inference attack against my diagnosis model at
http://localhost:8009/predict. Here are my known members and non-members with
labels - report the AUC, TPR at 1% FPR, and how many records were re-identified.

From the CLI:

# shadow_model trains shadow models; threshold, entropy, and lira are alternatives.
dn airt run-classifier --attack shadow_model \
  --endpoint http://localhost:8010/predict --num-classes 10 --modality image

The target must expose /pool, /members, and /nonmembers data helpers (as the demo targets do); pass --api-key if the endpoint requires an x-api-key.

A high non-member score is a false positive

The finding scores every record. A non-member that scores above the threshold is a false positive, not a re-identification - the attack guessed wrong. Read the outcome column, not the score, before you call a record leaked.

Read the finding

The finding answers one question: can someone prove a specific record was in your training set?

• Membership AUC - how well the signal separates members from non-members across all thresholds. 0.5 is a coin flip; above ~0.7 the model leaks.
• TPR at 1% FPR - the attacker’s true-positive rate when they accept almost no false alarms. This is the number that matters: it says how many real members they catch while being nearly certain each hit is real.
• Attacker advantage - how far above chance the attack performs.
• Attack accuracy, precision, recall - the quality of the member vs non-member call at the decision threshold. Precision says how many of the flagged records really were members (low precision means false accusations); recall says how many real members were caught.
• Records re-identified - the count of members correctly caught at the operating threshold.
• Per-record leak table - each row shows the actual record, the attack’s verdict, and an outcome: re-identified for a member caught, false positive for a non-member wrongly flagged.
• ROC curve - the full trade-off between catching members and false alarms.
• Member-vs-non-member score distribution - the two histograms. The more they separate, the worse the leak.

You are exposed when AUC is high and TPR at 1% FPR is meaningfully above 1% - an attacker proves membership for named individuals with high confidence. You held when the two score distributions overlap, AUC sits near 0.5, and TPR at 1% FPR stays at the noise floor.

Mitigations

Apply these in order, cheapest and highest-leverage first:

1. Reduce overfitting. Regularization, early stopping, and more training data shrink the confidence gap between members and non-members that the attack reads.
2. Coarsen outputs. Return top-1 or bucketed confidence so the per-record signal disappears.
3. Train with DP-SGD. Differential privacy bounds how much any single record shifts the model, cutting membership advantage toward chance at some accuracy cost.
4. Rate-limit and monitor. Membership scoring probes members and non-members in volume; throttle and alert on that access pattern.

The expanded finding shows the member-vs-non-member score distributions, the AUC, and a table of re-identified records - each row is one individual the model confirmed was in its training set.