Frank

Appearance

frankctl pipelines apply -f — error cookbook ​

What the common failures look like, why they happen, and how to recover. Anchored on the apply path that landed in PRs #462, #465+#466, #467+#468 (master tracker #457).

Pre-flight before you read this page ​

Before your apply hits any of these errors in anger:

bash
frankctl pipelines apply -f vertical.yaml --dry-run-server

The server-side preflight POSTs your multi-doc body to /api/v1/apply/dry-run, runs the same pattern-catalog lookup, name-resolution, and immutable-diff checks the real apply does, and returns a per-doc verdict (create | patch | noop | would-409 | error) without persisting anything. Operators catch ~80% of the errors below this way without touching any real DB state.

--dry-run (no suffix) is the older client-only check — just YAML parse + dep-order. Use --dry-run-server for the deeper validation.

How to read this page ​

Every error has:

  • Signature — what the CLI / server prints.
  • Why it happens — what state on the server caused it.
  • Fix — copy-paste recovery, typically a CLI invocation.

Errors are grouped by the exit code class (auth, validation, collision, etc.). If your error doesn't match, check the JSON body from the server (frankctl ... --json); the apply path always returns a structured {detail: ...} body.

immutable_diff — 409, the most common stumble ​

Signature ​

failed at Source/owm_air_pollution: immutable_diff
  fields:
    - {name: pattern_id, existing: "rest_api", requested: "http_paginated"}

Why ​

?if-not-exists=true (which apply -f always sends) found an existing row at the lookup key and the request wants to change an immutable field. Immutable fields per kind:

  • Source — pattern_id
  • Pipeline — tenant_id only (everything else PATCHes)
  • BackingDataset — iceberg_namespace, iceberg_table, entity_type_name, schema_library_ref, schema_version, sync_mode, transform_id, pipeline_id

The lookup keys are not themselves "immutable" in the diff sense — changing them means looking up a different row, not failing.

Fix — preferred: revert the YAML ​

If you didn't mean to change the immutable field, revert your YAML and re-apply. apply -f is idempotent; no DB state changed.

Fix — when you actually want to recreate ​

Operational state on the existing row (Source.cursor, source_loaded_files, BackingDataset.last_sync_snapshot_id, last_synced_at, etc.) WILL BE LOST.

bash
frankctl pipelines apply --allow-recreate -f vertical.yaml

The CLI receives the 409 body (which carries existing_id), DELETEs the row, and re-POSTs the new shape. Cascade DELETE applies — child rows pointed at this Source/Pipeline/BD get cleaned up by Frank's existing DELETE handlers.

Fix — for tenant_id ​

You can't change tenant_id ever. It comes from your auth token. If you're seeing a tenant_id diff, you're logged in as the wrong tenant. Run frankctl auth login again with the right realm.

narrowing — 409 on ensure_schema: ​

Signature ​

failed at BackingDataset/air_quality_observed:
  ensure_schema rejected — narrowing
  removed: [pm10]

Why ​

Your ensure_schema: block doesn't list a field the live ontology schema currently has. Frank treats that as a deprecation request and refuses by default.

Fix — preferred: add the field back ​

If you didn't mean to remove the field, add it back to ensure_schema: and re-apply.

Fix — when you actually want to deprecate ​

bash
frankctl pipelines apply --allow-deprecate -f vertical.yaml

Frank calls publish_version(deprecate_fields=[pm10]). The field is marked deprecated; existing rows keep their values; new rows can't set it. See docs/reference/ensure-schema.md for the full semantics.

field_type mismatch — 409, NO escape hatch ​

Signature ​

failed at BackingDataset/air_quality_observed:
  ensure_schema rejected — field_type mismatch on `pm2p5`
  declared:  {"type": "integer"}
  live:      {"type": "number"}

Why ​

You changed the declared field_type for a field that already exists in the live schema. Type changes are NEVER allowed via apply — neither --allow-deprecate nor --allow-recreate bypasses this.

Fix ​

You can't change a field's type. Add a new field with the new type and a different field_key:

yaml
ensure_schema:
  fields:
    - { field_key: pm2p5,     field_type: { type: number } }     # keep the old
    - { field_key: pm2p5_int, field_type: { type: integer } }    # add the new

Then in silver, map the value to the new field. Once consumers migrate, deprecate the old field with --allow-deprecate.

See docs/reference/ensure-schema.md § "Type changes" for the full migration playbook.

Pattern '<id>' not found — 400 ​

Signature ​

failed at Source/owm_air_pollution:
  Pattern 'rest_api' not found

Why ​

The Frank deployment doesn't have the named pattern in its catalog. Either:

  • You're on an older Frank deployment that pre-dates the pattern.
  • The pattern catalog hasn't been seeded (fresh deploys need make seed-pattern-registry or equivalent).
  • You typo'd the pattern_id.

Fix ​

bash
# Confirm what patterns exist:
frankctl patterns list

# If the pattern's missing, ask the operator who runs the deploy to
# seed the registry. If the pattern ID was a typo, edit the YAML.

Source named 'X' already exists — 409 WITHOUT ?if-not-exists ​

Signature ​

failed at Source/owm_air_pollution:
  A source named 'owm_air_pollution' already exists. Choose a different name.

Why ​

You hit the soft-delete edge case OR you're using an older CLI that doesn't send ?if-not-exists=true.

Fix ​

bash
# Confirm your CLI version:
frankctl --version    # should be >= the version that shipped #467

# If old, upgrade. Modern CLI always sends ?if-not-exists=true.

# If new, the row is soft-deleted and the partial unique index is
# blocking re-create. The soft-delete partial unique index landed in
# migration s31. Run a hard-delete or pick a new name:
frankctl sources delete <id-of-soft-deleted> --yes

"Sources/Pipelines that succeeded stay in the database" ​

Signature ​

applying Source/owm_air_pollution... ✓
applying Pipeline/air_quality_observed... ✗ failed (...)

Why ​

The CLI applies docs in dependency order (Source → Pipeline → BD). A failure mid-file leaves earlier docs in the database. This is intentional — the spec calls it "no partial-apply on the failing doc". The earlier docs are still applied successfully.

Fix ​

Fix the failing doc and re-apply. The successful earlier docs will no-op on the next pass (200 PATCH or unchanged).

If you want to roll back the successful ones too:

bash
# Pick them off manually:
frankctl sources delete <source-id> --yes
frankctl pipelines delete <pipeline-id> --yes

# Or take a wider scoop:
frankctl sources delete --filter name=owm_air_pollution --yes

bronze_field_types_read_failed — warning, not an error ​

Signature (in the worker log, not user-facing) ​

{"event": "bronze_field_types_read_failed", "table": "bronze.x.y", "error_type": "..."}

Why ​

#469's self-aware writer tried to read bronze's existing column types to build a coercer pipeline, and the Iceberg catalog read failed (auth, network, permissions). The sync isn't blocked — Frank falls through to the legacy defensive int→float promotion. Brand-new tables also log this (the table doesn't exist yet).

Fix ​

If the table exists and the read fails persistently, check Iceberg catalog credentials. Otherwise ignore — the sync still runs.

"AI service unavailable (Martha offline)" ​

Signature ​

AI service unavailable (Martha offline). Skipping.

Why ​

frankctl ai compose-pipeline (or any other ai verb) couldn't reach Martha. Martha is the external AI orchestration service; if it's down, every AI endpoint returns {available: false} and the CLI exits with code 4.

Fix ​

  • Wait. Martha outages are usually short.
  • Author the YAML by hand. The CLI never depends on AI to do the declarative apply path — ai compose-pipeline is a convenience, not a requirement.

Type-drift sync failures (Cannot change column type) ​

This isn't an apply-time error — it's a sync-time error after the BD is in place. But it's the most common "why is my data not landing" question post-apply, so it lives here too.

Signature ​

All 1 batch(es) failed: Cannot change column type: dt: long -> double

Why ​

The extracted record value's type doesn't match what bronze stores. See the full self-aware-writer architecture in #469 — bronze should auto-resolve most of these, but operator-pinned types via set-types still apply.

Fix ​

bash
frankctl sources streams diff-types <source-id> <stream-id>
frankctl sources streams set-types <source-id> <stream-id> --field dt=integer

See docs/guides/cli.md § "Bronze type drift" for the full playbook.

Connection / 5xx errors ​

Signature ​

Server error (500). Run `frankctl status` to check API health.

Why ​

Genuinely a server-side issue. Frank's apply routes are robust against the common operator mistakes; 5xx means something broke server-side.

Fix ​

bash
# 1. Is the API healthy?
frankctl status

# 2. If yes, the route was momentarily bad. Re-run apply.
#    The path is idempotent.

# 3. If the same 5xx repeats, file an issue with the trace_id from
#    the response (every Frank response carries one).

See also ​

  • docs/guides/cli.md — full reference for pipelines apply -f.
  • docs/reference/ensure-schema.md — ensure_schema: semantics.
  • docs/reference/patterns.md — pattern catalog.
  • docs/getting-started/first-vertical.md — happy-path tutorial.

Frank — low-code EL/T for the lakehouse.