Assessments

Assessments organize AI red teaming campaigns with DreadAIRT. An assessment is a named container that groups attack runs against an AI system and aggregates their results into analytics, findings, and compliance reports.

What an assessment is

An assessment answers: How vulnerable is this AI system to adversarial attacks?

You provide:

• a target system to probe
• one or more attack strategies (TAP, GOAT, Crescendo, PAIR, and others)
• goals describing what the attacks should attempt

Dreadnode executes attack runs through its red teaming workflows and aggregates their telemetry into analytics on demand.

An assessment belongs to a project within a workspace and accumulates results across multiple attack runs over time.

Key concepts

Concept	Definition
Assessment	A named, project-scoped container for a red teaming campaign
Attack Run	A single execution of an attack strategy (e.g., one TAP run with a specific goal)
Trial	An individual attempt within an attack run — one conversation or prompt exchange with the target
ASR	Attack Success Rate — the fraction of trials that achieved the stated goal
Risk Score	A composite metric combining ASR, severity, and attack effectiveness
Transform	An adversarial technique applied to prompts (encoding, persuasion, injection, etc.)
Compliance Tag	A mapping from attack results to security framework categories (OWASP, MITRE ATLAS, NIST, Google SAIF)

Execution flow

The assessment lifecycle spans the operator workflow and the platform:

1. Operator workflow — launch attacks against a target and capture structured telemetry
2. Platform — ingest telemetry and materialize analytics on demand

Note

The platform organizes and analyzes results while the attack workflow drives execution. See AI Red Teaming for a TUI-first workflow.

Analytics and reporting

The platform provides several levels of analysis:

Assessment-level:

• Aggregated trace statistics (total attacks, trials, ASR, risk scores)
• Per-attack span breakdowns with success rates and severity
• Individual trial spans with filtering by attack name, minimum score, and jailbreak status

Project-level:

• Cross-assessment findings with severity, category, and attack name filtering
• Executive summary with risk trends, compliance posture, and top vulnerabilities
• Automated report generation combining findings across all assessments in a project

Compliance mapping: Results are tagged against industry security frameworks:

• OWASP Top 10 for LLM Applications
• MITRE ATLAS
• NIST AI Risk Management Framework
• Google Secure AI Framework (SAIF)

Reports

Reports are generated from assessment or project data and persisted for later retrieval. A report captures a point-in-time snapshot of findings, risk scores, and compliance posture. Reports can be generated at both the individual assessment level and the project level (consolidating across all assessments).