White-box metrics & tuning

White-box attacks (the GCG family, embedding-space, and gradient-free search) optimize an adversarial suffix against a local open-weight surrogate. That optimization leaves a trail - a per-step loss curve and the hyperparameters it used - which the platform captures on the resulting finding. This page explains how to read it so you can tell why a run succeeded or stalled and what to change next.

Where it shows up

Expand a white-box finding in the Assessments → Findings view. Alongside the attacker prompt and target response you get a White-box optimization card and an Attack-loss convergence chart. The card only appears for white-box findings; black-box jailbreaks do not have a surrogate loss.

The convergence chart

The chart plots the attack loss on the surrogate against the optimizer step. Loss is the cross-entropy of the affirmative target response (e.g. "Sure, here is"): lower means the optimizer drove the surrogate closer to complying, which is what transfers to your real target.

• Falling then flattening - healthy convergence; the suffix is doing its job.
• Still falling at the last step - the run was cut off by the step budget. Raise n_iterations.
• Flat from the start - the surrogate, target affirmation, or search width is wrong (see tuning below).

A dashed grad norm series is overlaid for gradient attacks (GCG family); gradient-free attacks (adaptive random search, generators) omit it.

The optimization card

Field	What it tells you
Optimizer	Which white-box algorithm ran (gcg, adaptive_random_search, …).
Surrogate model	The local open-weight model whose signal drove the attack.
Access	transfer (optimized on a surrogate, sent to a black-box target) or direct (surrogate attacked).
Best / final loss	Lowest and last loss reached. A high final loss that is still dropping means the budget ran out.
Acceptance rate	Fraction of steps that found an improving candidate. Low means the search is starved.
Steps run	Steps executed vs budget. Fewer than the budget means early stopping fired (the attack succeeded).
Knobs used	control_len, topk, n_candidates, n_iterations, lr - the settings this run used.

The card also prints a plain-language tuning hint derived from these signals.

Tuning the next run

Symptom	Change
Loss still falling at the budget	Raise n_iterations.
Low acceptance rate (< 20%)	Raise topk / n_candidates so each step has more options to accept.
Loss plateaus high and never converges	Try a longer control_len, a different surrogate, or a new target affirmation.
Converges fast but does not transfer	Use a surrogate closer to the target family; lower early-stopping if over-fitting the surrogate.

Sample payload

The same values are stored on the finding’s metrics_detail (and exportable), so you can chart or threshold them yourself:

{
  "metrics_family": "whitebox",
  "optimizer": "gcg",
  "surrogate_model": "Qwen/Qwen2.5-0.5B-Instruct",
  "access_level": "transfer",
  "loss_curve": [
    [0, 1.349],
    [1, 1.183],
    [10, 0.866]
  ],
  "grad_norm_curve": [
    [0, 2.19],
    [1, 2.25]
  ],
  "best_loss": 0.866,
  "final_loss": 0.866,
  "loss_still_falling": true,
  "acceptance_rate": 0.83,
  "n_steps": 11,
  "n_iterations": 50,
  "control_len": 20,
  "topk": 256,
  "n_candidates": 512,
  "final_suffix": "!];//Bootstrap/moment"
}

Per-step detail lives in Traces

The card summarizes the run. For the full per-step span stream (every candidate batch, loss, and accepted swap) open the Traces tab - each step emits a whitebox_step span.