Architecture

This page separates what you can run in the current preview from the product's target logical architecture. The preview is an implemented, narrow path; the target is design direction and does not mean every component is available or production-ready.

Current preview: one runnable cross-source path

The Quickstart captures MySQL orders and PostgreSQL shipments, uses nest to assemble one order document, and materializes the result in its managed MongoDB store:

Control path

.tap.yml workspace

Source and pipeline resources.

validate / apply

tapstate CLI

Offline authoring and authenticated control requests.

Submit (HTTP)

Data path

MySQL + PostgreSQL

Independent initial snapshots and later CDC.

Single-node server

Capture & transform

Registers artifacts, runs pipelines, and reports status.

Managed MongoDB state

The Quickstart materializes its view here.

Current preview boundaries

• The native CLI scaffolds and validates *.tap.yml resources offline.
• A connected CLI session can register connector artifacts, apply resources, start a pipeline, and inspect status, metrics, and logs.
• The CLI, REST API, and MCP provide bounded preview inspection, not a stable application-facing data API.
• The runtime and CLI are prerelease surfaces; command and API compatibility can change before a stable release.
• The pipeline DAG wires filter, map, js, union, and nest.
• nest maintains a document assembled from related source tables. The v1 Schema also declares join, but the current DAG builder refuses it.
• An inline view materializes the Quickstart result in the managed store. serve.sync is the separate path for delivery to an external target. serve.query and serve.push are schema-defined declarations that the current preview does not execute.

Recovery and scale limits

• The server is single-node and uses in-memory runtime state.
• A server restart replays from the source instead of resuming a persisted source offset.
• High availability and production recovery guarantees are not part of the current preview contract.

See Create your own workspace for the offline authoring path. The Quickstart packages this preview topology for local exploration; it is not a deployment architecture.

Target logical architecture

The target design expands this one runnable path into a general data plane and control plane. It explains the intended product boundaries, not additional current-preview components.

Control plane

Author

Define connections and pipelines.

Validate

Check resources, references, and connector fields.

Operate

Apply, observe, and control lifecycle.

Data plane

Sources

Databases, brokers, files, and APIs.

Capture

Initial data and later changes.

Transform

Filter, map, script, union, and assemble related records with nest.

Materialize

Maintain destination-ready current state.

Deliver

Write targets or publish streams.

Consumers

Applications, APIs, and agents.

Durable recovery state

Resource versions · checkpoints · schema and mapping state · retries · operational history

Target data plane

Capture establishes an initial state and follows later changes. Transform applies reviewable shaping and document-assembly rules. Materialize maintains consumer-oriented state, and Deliver exposes that state through connector or query surfaces.

The target recovery boundary owns source positions, checkpoints, retry state, schema state, and operational history durably. This is a design requirement, not a guarantee of the current in-memory preview.

Target control plane

The target control plane covers versioned intent, lifecycle and authorization, status, diagnostics, metrics, audit information, and human- and agent-facing interfaces over the same deterministic operations.

An AI assistant is a client of those controls, not the source of product truth. Schema, catalog metadata, runtime state, and authorization decisions remain authoritative within their own boundaries.

External-system boundary

Source log retention, network paths, credentials, target constraints, API rate limits, and broker retention remain part of the deployment. Prepare and monitor them alongside the tapstate data path.

Use connector guides for external-system preparation and Troubleshooting to isolate authoring, deployment, and data-movement failures.