Skip to main content

Manifest backup and restore

The config manifest is the entire control-plane configuration serialized as one YAML document: every tenant, database, pool, federated source, role, group, user, and grant. Export it to back up or version-control your configuration, then import it to restore, clone an environment, or apply a reviewed change. The manifest holds configuration only; it does not contain table data or the DuckLake catalogs.

Manifest export and import are cross-tenant operations and are restricted to superusers. A tenant-scoped session is rejected with 403 superuser_required; a static QOD_API_KEY caller is admitted. The examples below use the qod CLI; they assume qod login has stored a superuser session, or QOD_API_KEY is set for CI scripts.

Export​

qod manifest export --out manifest.yaml

GET /api/manifest/export returns the manifest as application/yaml. The top of the document records provenance:

apiVersion: quack-on-demand/v1
kind: ConfigManifest
exportedAt: 2026-06-10T12:00:00Z
exportedFrom:
  managerVersion: 0.3.2
  hostname: qod-1

What the export contains​

The document mirrors the object hierarchy:

  • tenants[] - each with its authProvider / authConfig, and nested:
    • tenantDbs[] - kind, metastore, dataPath, objectStore, defaultDatabase, defaultSchema, and nested federatedSources[] (with their secrets[]).
    • pools[] - tenantDb, roleDistribution, maxConcurrentPerNode, disabled, and optional cohorts[] placement.
    • identities[] - external identity mappings for the tenant.
  • roles[] - each (tenant, name) with its permissions[] (catalog/schema/table/verb).
  • groups[] - each (tenant, name) with its assigned roles[].
  • users[] - tenant (omitted/null for a superuser), username, passwordHash, role, enabled, mustChangePassword (optional, default false), and the user's roles[], groups[], and poolGrants[].

Sensitivity of an export​

Plaintext credentials are never written on export, but the file is NOT free of credential material:

  • User password (plaintext) is omitted entirely from exported users; there is no plaintext to export, only bcrypt hashes are stored.
  • User passwordHash carries the user's real bcrypt hash verbatim. This is what lets a backup restore the same credential without anyone re-typing passwords.
  • Federated secret values are written as ***REDACTED***; an externalRef is written verbatim.
  • A typed federated source's config (see External Iceberg catalogs) is written verbatim. Its credential fields hold {{secret.NAME}} placeholders rather than values, so the credentials themselves are covered by the rule above, but every other field may hold a literal: a uri carrying userinfo or a signed query parameter is exported in the clear. This is the same policy the manifest already applies to setupSql, which can hold a full connection string. Redacting uri was considered and rejected, because a round trip has to reproduce an attachable catalog and a redaction sentinel there imports as a source that can never attach.
  • A database's encryptionKey (see Encryption at rest) is redacted. The encrypted flag itself round-trips, so the manifest still records that the database is encrypted, but it cannot recreate an encrypted DuckDB file database on another deployment. Re-applying to the SAME deployment is unaffected: the stored key is carried forward.

A database's metastore is exported verbatim, including its control-plane password. For an encrypted DuckLake database that password opens the catalog holding every per-file key, so an export is a decryption credential for that database however its own key is handled.

Because bcrypt hashes of weak passwords are subject to offline cracking, treat an exported manifest as sensitive: do not commit it to a public repository, and store it with the same care as any other credential material (private repository, secret store, or encrypted backup).

A plain export-then-import round-trip preserves credentials without rotation: exported hashes are applied verbatim on import, an absent hash falls back to the existing row, and redacted secret values are reused from the existing rows (see below).

Import​

qod manifest import manifest.yaml

POST /api/manifest/import takes the YAML body and returns a summary of how many top-level resources were in the manifest:

{"tenants":2,"tenantDbs":3,"pools":4,"roles":5,"groups":2,"users":7}

Import validates the whole document before writing anything. On failure it returns 400 and changes nothing:

  • invalid-yaml - the body did not parse as YAML.
  • invalid-manifest - validation failed: a wrong apiVersion (must be quack-on-demand/v1), duplicate keys (tenant name, (tenant, role), (tenant, group), (tenant, user), or a nested duplicate), or a user/role/group that references a tenant not present in the manifest or already in the database.

After a successful import the manager reloads its in-memory caches (tenants, databases, pools, RBAC effective sets) immediately. No restart is needed for the new configuration to take effect.

Apply semantics​

This is the part to understand before importing a hand-edited file. The rules differ between top-level and nested resources:

  • Top level is additive (upsert only). Tenants, roles, groups, and users that appear in the manifest are created or updated. Top-level resources that are absent from the manifest are left untouched: importing a manifest that lists only tenant: acme does not delete tenant: widgets. None of the top-level lists are required, so a manifest may omit users, roles, or groups entirely. An omitted user is ignored, never deleted; the importer only ever upserts the users it finds in the file.
  • Nested collections under a present parent are replace (delete-then-upsert). For any parent that is in the manifest, its child collections are made to match the manifest exactly. Children present in the manifest are upserted; children that exist in the database but are missing from the manifest are deleted.
Nested omissions delete

Because nested collections are replaced, omitting a child under a parent you do include removes it. If you export, then hand-edit a tenant to drop one pool from its pools list and re-import, that pool is deleted. A role's permissions are fully replaced; a user's poolGrants are fully replaced. To change one tenant safely, keep all of its databases, pools, roles, and grants in the file, or edit a full export rather than a partial document.

Replaying an old manifest over an Iceberg source​

A manifest is a statement of desired state, so import applies a federated source's sourceType as written. That includes changing it. The REST and CLI paths refuse an in-place flip between sql and iceberg_rest, because it would silently replace operator-written setupSql with a rendered config or the reverse; import does not, and applies the manifest's readOnly along with it.

The case that costs you something is replaying a manifest exported before an alias was converted to an Iceberg source. The alias goes back to being a sql source, and because a sql source's readOnly defaults to false, a catalog that was read-only becomes writable at the next node spawn.

This is intended for a declarative apply, but it is never silent. The importer emits a WARN naming the database, the alias and the transition, and it is the only audit line the downgrade leaves:

manifest import: tenant-db 'acme_lake' federated source 'sales_lake' changes sourceType 'iceberg_rest' -> 'sql' (readOnly becomes false); REST refuses this transition in place

Before re-applying an old manifest, check it against a current export for any federatedSources entry whose sourceType has moved backwards, and grep the import for that line afterwards.

Passwords and secrets on import​

  • A user with a passwordHash value has it applied verbatim (no re-hashing): this is the field an export populates, so a round-trip restores the exact credential the user had at export time.
  • Otherwise, a user with a password value sets it (the value is treated as a bcrypt hash if it looks like one, otherwise as plaintext to be hashed). A user with neither field reuses the bcrypt hash captured from the existing row at the start of the import, so hand-written manifests that omit both keep existing users' passwords unchanged.
  • A federated secret left as value: "***REDACTED***" (and no externalRef) reuses the existing stored value. This is why a redacted export re-imports without re-typing credentials.

Typical workflows​

  • Backup. Export on a schedule and store manifest.yaml somewhere access-controlled (private repository, secret store, or encrypted backup). The file carries no plaintext credentials, but it DOES carry each user's bcrypt password hash, so treat it as sensitive and keep it out of public repositories.
  • Promote a change. Export, edit in a reviewed pull request, re-import. Validation plus all-or-nothing apply makes a bad edit fail before it touches the store.
  • Clone an environment. Import a source environment's manifest into a fresh manager. Exported passwordHash values carry user credentials over as-is; provide real password / secret values only for anything the manifest does not carry, since there is nothing to reuse on a first import.

For the objects the manifest carries, see "Tenants and databases", "Pools and cohorts", Federation, and the Access control model.