Patterns Reference

Patterns are how Frank expands capability without rewriting the product. They describe source connectors, transform actions, runtime contracts, UI forms, and validation rules in structured files.

There are two pattern families:

Family	Location	Purpose
Source patterns	backend/config/patterns/*.json	Configure extract/load sources and discovery forms.
Transform patterns	backend/config/transform_patterns/*/pattern.yaml	Configure reusable transform actions, params, renderers, and Python-runner packages.

Source patterns

Source patterns drive the source creation wizard and source discovery APIs.

Core fields:

Field	Meaning
id	Stable pattern ID such as postgres or rest_api.
name	User-facing name.
description	Short explanation.
category	UI grouping such as database, api, file, crm, finance, streaming.
complexity	UI hint for setup difficulty.
extends	Optional parent pattern for inheritance.
engine	airbyte or dlt.
airbyte_connector / dlt_source	Engine-specific connector metadata.
field_definitions	Required and optional config form fields.
config_defaults	Default source configuration.
config_templates	Reusable config shapes.
examples	Example use cases.
supported_formats	File or payload formats.
auth_methods	Supported auth schemes.
transformation_hints	Hints for later mapping and modeling.

Source pattern catalog

Installed source patterns include:

Pattern	Category	Engine
postgres	database	Airbyte
mysql	database	Airbyte
mssql	database	Airbyte
mongodb	database	Airbyte
bigquery	data warehouse	Airbyte
snowflake	data warehouse	Airbyte
redshift	data warehouse	Airbyte
databricks	data warehouse	Airbyte
salesforce	CRM	Airbyte
hubspot	CRM	Airbyte
google_ads	marketing	Airbyte
google_analytics	analytics	Airbyte
google_sheets	file	Airbyte
s3	file	Airbyte
sftp_bulk	file	Airbyte
rss	API	Airbyte
rest_api	API	dlt
graphql	API	dlt
github	API	dlt
jira	API	dlt
notion	API	dlt
slack	API	dlt
airtable	API	dlt
stripe	finance	dlt
filesystem	file	dlt
archive	file	dlt
kafka	streaming	dlt

Field definitions

Field definitions generate UI forms and validation expectations:

{
  "required": {
    "host": {
      "type": "string",
      "label": "Host",
      "description": "Database host"
    },
    "password": {
      "type": "password",
      "label": "Password"
    }
  },
  "optional": {
    "ssl": {
      "type": "boolean",
      "default": true
    }
  }
}

Supported field types include string, password, number, boolean, select, JSON object, and array-style values.

Source pattern inheritance

Patterns can extend another pattern. The registry merges parent and child config:

• Child connector config overrides parent connector config.
• Field definitions are merged.
• Defaults and templates are merged.
• Child pattern metadata remains the user-facing identity.

Use inheritance for specialized API patterns that share a base REST or file configuration.

Source pattern APIs

GET /api/v1/patterns/discover
GET /api/v1/patterns/{pattern_id}

CLI:

frankctl patterns list
frankctl patterns get postgres

Transform patterns

Transform patterns drive Quick Actions, reusable transform steps, and custom Python runner packages.

Core fields:

Field	Meaning
id	Stable transform pattern ID.
version	Pattern version.
name	User-facing name.
description	What the pattern does.
category	UI grouping.
complexity	Setup complexity.
params_schema	JSON Schema for pattern params.
input_contract	Required input shape.
output_contract	Output shape.
supported_runtimes	Runtime values such as trino_sql or python_runner.
ui_config	UI form hints.
example_params	Example param object.
is_active / is_deprecated	Lifecycle controls.

Transform pattern catalog

Installed transform patterns include:

Pattern	Category	Purpose
select_rename	projection	Select and rename columns.
filter	filtering	Filter rows with a SQL expression.
dedupe	deduplication	Remove duplicate records.
left_join	joining	Left join tables.
inner_join	joining	Inner join tables.
lookup_join	joining	Lookup enrichment.
aggregate	aggregation	Group by and aggregate.
window	analytics	Window functions.
validate_regex	validation	Validate text fields by regex.
validate_enum	validation	Validate categorical fields.
flag_anomaly	validation	Z-score anomaly flagging.
unit_convert	conversion	Unit conversion.
currency_convert	conversion	Currency conversion.
timezone_convert	conversion	Timezone conversion.
upsert	dimensions	Insert/update materialization.
scd_type1	dimensions	Type 1 dimension overwrite.
scd_merge	dimensions	Type 2 slowly changing dimension merge.
geo_parse_wkt	geospatial	Parse WKT geometry.
geo_contains	geospatial	Point-in-polygon checks.
geo_distance	geospatial	Distance calculations.
geo_nearest	geospatial	Nearest-neighbor matching.
h3_enrich	geospatial	H3 cell enrichment.
h3_aggregate	geospatial	H3 aggregation.
h3_spatial_join	geospatial	H3 spatial joins.
fx_rate_ingest	utilities	Python-runner FX rate ingestion.
python_echo	testing	Python-runner test pattern.

Transform pattern APIs

GET  /api/v1/transform-patterns
GET  /api/v1/transform-patterns/categories
GET  /api/v1/transform-patterns/{pattern_id}
POST /api/v1/transform-patterns/{pattern_id}/validate
POST /api/v1/transform-patterns/{pattern_id}/preview
POST /api/v1/transform-patterns/sync

CLI:

frankctl patterns validate-params filter -f params.yaml

Python-runner pattern structure

Python-runner patterns are authored as package directories:

my-pattern/
  pattern.yaml
  main.py
  requirements.txt
  Dockerfile
  tests/
    sample_config.json
    test_transform.py
  README.md

Use the Python frank CLI:

frank init my-pattern --template python
frank validate my-pattern
frank test my-pattern --config my-pattern/tests/sample_config.json
frank build my-pattern --tag my-pattern:v1

The runtime code should use SDK primitives:

• FrankContext.from_env()
• emit_log, emit_metric, emit_lineage, emit_progress
• FrankResult.success(...)
• result.write_to_stdout()

External pattern registration

Publishing flows can register pattern changes with Frank:

PATTERN_WEBHOOK_SECRET=... frankctl patterns register -f webhook-payload.json

The CLI signs the payload, and the API records or reconciles the pattern lifecycle. This is the path for generated or externally maintained transform packages to become available in the product.