Publishing

Publishing a dataset is two decisions: which bytes go into the registry, and who can see them. dn dataset push handles the upload; visibility is a separate, name-level switch you can flip at any time.

Push a version

dn dataset push ./support-prompts

Pushed acme/[email protected] (sha256:9ab81fc1...)

The CLI reads dataset.yaml, validates the manifest, hashes every artifact, uploads only the files the registry doesn’t already have, and registers the new version. Re-publishing a dataset with one added row only ships the delta.

Override the registry name

dn dataset push ./support-prompts --name intent-eval-set

Use --name when the directory and registry names diverge, or to publish into another org (--name another-org/intent-eval-set) you have write access to. Without --name, the name from dataset.yaml (or the directory name) is prefixed with your active organization.

Dry-run before uploading

dn dataset push ./support-prompts --skip-upload

--skip-upload runs every local step — schema validation, blob hashing, manifest build — and stops before the HTTP upload. Use it to verify the package cleanly in CI or when you want to know what will happen without committing bytes to the registry.

Publish directly from Python

import dreadnode as dn

dn.configure(server="https://app.dreadnode.io", api_key="dn_...", organization="acme")

result = dn.push_dataset("./support-prompts")
print(result.package_name, result.package_version)
# acme/support-prompts 0.1.0

dn.push_dataset accepts the same skip_upload and name arguments as the CLI. The returned PushResult carries manifest_digest, blobs_uploaded, blobs_skipped, and any errors.

Control visibility

Datasets are private to your organization by default. Visibility is name-level — every version of acme/support-prompts shares the same setting.

Action	Command
Make the dataset public	dn dataset publish support-prompts
Restrict it again	dn dataset unpublish support-prompts
Publish at push time	dn dataset push ./support-prompts --publish

publish and unpublish accept multiple names:

dn dataset publish support-prompts classify-intent

They reject version-qualified references ([email protected]) because visibility is not per-version. Flip the switch once and every version follows.

Caution

Public datasets are visible to every Dreadnode organization and appear in the shared catalog. Do not publish datasets that contain customer data, credentials, or anything you would not paste into a public GitHub repo.

Cut a new version

dataset.yaml

version: 0.2.0

dn dataset push ./support-prompts
# → acme/[email protected]

Each push requires a fresh, semver-valid version. The registry rejects collisions. Older versions remain accessible by their pinned references — downstream evaluations and training jobs that pointed at @0.1.0 keep working.

Downstream consumers don’t move until you update their references. Adopt @0.2.0 deliberately: update the evaluation manifest or training config, rerun, compare.

Retire a version

dn dataset delete acme/[email protected]

delete requires a version — there’s no “delete the whole family” verb. The CLI confirms before deleting; pass --yes for automation:

dn dataset delete acme/[email protected] --yes

Deletion is permanent. Evaluations, training jobs, and cached pulls that reference the deleted version will fail to resolve.

What to reach for next

• Make sure the bytes are right before you publish → Authoring
• Find it in the registry after publishing → Catalog
• Load it from Python or feed it into a training job → Using in code
• Every CLI verb → dn dataset