Troubleshoot a tapstate configuration

Start by identifying which layer produced the failure. A resource-file error, a connection failure, and incorrect destination data require different evidence and different fixes.

Read the command result

tapstate commands use these exit codes:

Use tapstate --help to inspect the commands in your installed version. In the current preview, use tapstate help <command> or tapstate <command> --help for a command's operands and options.

Text-mode results are written to standard output and usage or diagnostic messages are written to standard error. With -o json or -o yaml, the structured result, including validation diagnostics, is written to standard output. Preserve both streams and the exit code when collecting evidence.

CLI and server versions differ

The CLI and server are separate builds. When connect reaches a server on a different version, the CLI reports cli.version-mismatch and keeps the connection open. Run the following command in the same session to confirm both values:

version

Install the CLI version that matches the server, or connect to a server on the CLI version. If their contracts differ, the server decides which operations it accepts. Do not treat a successful connection as proof that every command is compatible.

If the CLI attempts anonymous issuer discovery against an older server that does not serve /.well-known/tapstate (such as a server on v0.3.0), the CLI reports cli.issuer-discovery-rejected. The diagnostic output includes:

• seed: The server endpoint URL contacted during discovery.
• reason: The rejection response or HTTP error status returned by the server.

To resolve cli.issuer-discovery-rejected, upgrade the tapstate server to a release that supports issuer discovery, or use a CLI version matching the older server.

Text-mode coded CLI diagnostics include the CLI version and the server version captured when the connection was made. Offline diagnostics say server not connected; an older server that does not report a version appears as server not reported. Structured JSON and YAML results keep their documented shape; collect tapstate version separately when reporting one.

Identify the failing layer

Do not use evidence from one layer as proof of another. A valid resource file is the starting point; connectivity and data movement still need to be verified in the environment where the connection runs.

Check the workspace first

Run the validation command supplied with your tapstate environment before investigating the external system:

tapstate validate --workdir tapstate-work

Review the filename, location, diagnostic code, message, and suggested fix. Correct one layer at a time, then validate again.

Demo workspace already exists or cannot be written

tapstate demo protects existing files by default:

• cli.demo-workspace-exists means the target already contains a generated demo. Choose an empty work directory, or inspect the existing files before retrying with --force.
• cli.workspace-not-writable includes the failing path and filesystem reason. Correct the permission or choose a writable directory. The command restores replaced files and removes files it created, although empty directories from the failed attempt can remain.

The tap alias is unavailable

The optional alias never replaces another command. cli.alias-name-taken means the target path belongs to something else. Continue using tapstate, or remove the conflicting command yourself before running tapstate alias install. cli.alias-install-dir-unknown means the selected installation directory has no tapstate entry; set TAPSTATE_INSTALL_DIR to the directory used by the installer and retry.

Unknown field or invalid value

• Confirm the connector ID and exact field spelling in its Reference section.
• Remove fields copied from an upstream UI or another connector version unless the tapstate contract exposes them.
• Check whether the field is conditional on another value, such as authentication method or storage type.

Invalid mode

• Use only modes shown on the connector page.
• Do not add cdc because the underlying database supports a transaction log.
• A target-only connection does not need a source read mode.

Missing or unresolved reference

• Confirm that pipeline references match resource IDs exactly.
• Keep every referenced resource in the selected work directory.
• Check for renamed files whose internal id did not change with the filename.

Row-expression schema and type errors

These checks apply to filter expressions and computed map fields that read after.<field> or before.<field>:

• dsl.row-expression-needs-discovery means at least one source that can reach the expression has no discovered schema. Apply the source connection by itself, run discover-schema <source-id>, then apply the pipeline or complete workspace again.
• dsl.row-expression-type-unsupported means the referenced column cannot be represented in CEL without losing information. Carry the column through unchanged, or compute from a compatible column instead.
• dsl.row-expression-type-unknown means the discovered schema has no resolved type for the referenced column. Remove that reference or use a column with a declared type.

These diagnostics restrict computation, not pass-through. A column that cannot be computed losslessly can still flow through a map when no computed rule reads it.

Upsert source has no discovered primary key

dsl.upsert-needs-key is a server-side apply diagnostic. It means a selected source table is being written with upsert, but its discovered schema has no primary key. The diagnostic identifies the affected source, table, and resource path.

Discover the source again and select a table with a primary key. A unique index, a target-side key, or tables[].pk does not satisfy this runtime check. Use append only when the delivery is insert-only and source updates and deletes do not need keyed handling.

Dotted field names in document stores

schema.column-name-reads-as-a-path is an apply-time advisory warning. It indicates that a source table column contains a dot (such as price.usd) and is being mapped to a document store like MongoDB.

Document stores write literal dotted keys, but query paths interpret dots as navigation into nested documents, causing direct field queries to return empty results and preventing index creation. To resolve this, rename the column in the source database or insert a map transform step to alias the dot to an underscore before writing to the destination.

Nest and Join transform diagnostics

Troubleshoot the Quickstart

The Quickstart assembles five MySQL orders and six PostgreSQL shipments into MongoDB. Treat the run as successful only when it prints this complete result:

quickstart: 5 orders from MySQL, each carrying its shipments from PostgreSQL -- 6 of them, in one object per order

A healthy server or order_pipeline running line does not establish that the assembled state is correct. Inspect the generated document when the success line is missing.

Docker is unavailable

docker info
docker compose version

Start Docker and confirm the Compose plugin is available. The Quickstart needs to pull and run MySQL, PostgreSQL, MongoDB, and the tapstate server.

A container is unhealthy or stopped

docker compose ps
docker compose logs --tail=200 mysql postgres mongo server

Wait for MySQL and PostgreSQL to finish initialization before retrying source schema discovery. A healthy tapstate server does not mean either source is ready. Slow Docker storage can leave a database initializing while the server already accepts requests.

For cli.request-timed-out during connector registration, check the registered catalog before retrying: the upload can finish on the server after the client times out. Registration is idempotent. Resolve storage or memory pressure, then retry the failed steps. Keep passwords and tokens out of shared log bundles.

The assembled result does not appear

Check the source and target independently:

docker compose exec mysql mysql -uroot -psecret appdb \
  -e "SELECT COUNT(*) AS source_rows FROM orders;"

docker compose exec postgres psql -U postgres -d appdb \
  -c "SELECT COUNT(*) AS source_rows FROM shipments;"

docker compose exec mongo mongosh --quiet \
  'mongodb://mongo:27017/views?directConnection=true' \
  --eval 'printjson({orders: db.order_state.countDocuments({}), shipments: db.order_state.aggregate([{$unwind:"$shipments"},{$count:"count"}]).toArray()})'

If MySQL has five rows and MongoDB does not, enter the CLI REPL and collect pipeline evidence:

./tapstate -w work

connect http://127.0.0.1:8080
login admin
status order_pipeline
metrics order_pipeline
logs order_pipeline

Enter the password stored in the local .env file. Do not paste it into a shell command or diagnostic bundle.

A rerun reports an illegal lifecycle transition

The script preserves generated assets. Inspect status and logs first. If the sample is already running and the five-order, six-shipment check passes, the setup is complete.

For a failed pipeline, fix the reported cause, stop it with --keep-state, and wait for the stop to complete before retrying the saved script from the same directory. See Lifecycle transitions. A lifecycle error alone is not a reason to delete the playground's volumes.

The managed view store is unreachable

actuation.view-store-unreachable means the server cannot reach the managed MongoDB store used for an inline view. Check the server's view-store configuration and MongoDB health, then start the pipeline again. The Quickstart's unauthenticated sample store is disposable and is not a production security model.

A connected command returns exit code 3

The command was invoked without a server connection, or the verb is reserved but not implemented. Start an interactive session and run connect and login, or seed a one-shot command with -c and -u. For automation, pass the password through TAPSTATE_PASSWORD instead of -p so it is not exposed in shell history or the process list.

Connector registration is refused or download fails

• connector.not-official means the server refused the artifact before registration. The default accepted set has 17 IDs across MySQL, PostgreSQL, MongoDB, Oracle, SQL Server, and Db2, including selected managed variants. Check the exact accepted IDs; a catalog entry alone does not make an artifact acceptable or certify every role.
• connector.artifact-unreadable (HTTP 400) means the uploaded JAR archive or its specification cannot be read. The error's artifact parameter names the file; the connector ID may not yet be known. Check that the download is a complete JAR, not an HTML error page, then fetch the matching artifact again before retrying registration.
• cli.connector-download-failed means register <connector-id> could not fetch the artifact from the GitHub preview release or the mirror configured in TAPSTATE_CONNECTORS_URL. Verify network connectivity, HTTPS proxy settings, or set TAPSTATE_CONNECTORS_URL to an internal mirror.

Register connectors one at a time. v0.7 still has a known ambiguity during concurrent registration; inspect the resulting catalog before retrying.

Unsupported target connector

dsl.unsupported-target-connector means a serve.sync destination is not allowed by this deployment's target rules. Cloud accepts MongoDB and MongoDB Atlas targets. Self-hosted deployments use the registered connector's sink capability, subject to source-only restrictions. Db2 and aws-rds-mysql are source-only in both modes; aws-rds-mysql is refused as a target even when its catalog entry reports sink capability. Check the destination's exact connector ID and registration rules. Catalog capability does not establish release certification.

Check external-system preparation

If the resource shape is valid but a deployment cannot connect or discover objects:

1. Confirm DNS, port, routing, firewall, and allowlist rules from the runtime host.
2. Authenticate with the same account outside tapstate when a safe vendor tool is available.
3. Recheck source and target privileges separately.
4. Verify the database, schema, warehouse, tenant, topic, bucket, or API scope.
5. Follow the connector page's mode-specific preparation path.

For TLS and connection testing, use only the fields and workflow supported by the tapstate environment you operate. Do not copy certificate fields or test commands from another product.

Check Snapshot or CDC behavior

Table selection errors

Table selectors are resolved when the runtime builds the pipeline. Validate the resource first, then run schema discovery for the source and apply the updated resources before starting again.

For an incomplete Snapshot:

• compare the selected object list with source discovery;
• check table-pattern and include/exclude rules;
• compare counts at a stable source boundary;
• inspect unsupported objects and data types in the connector limitations.

For CDC that stops or falls behind:

• check log, WAL, Oplog, snapshot, or broker retention;
• confirm that the saved recovery position still exists;
• verify replication or change-capture privileges;
• inspect long transactions and source-side cleanup jobs;
• test insert, update, delete, stop, and resume in staging.

PostgreSQL reports an active replication slot

If a capture failure says that its replication slot is already active, check pg_replication_slots.active_pid and the owning database session. Stop the affected pipeline and confirm that the previous reader has disconnected before starting another reader. Do not drop a slot that an existing pipeline needs.

After recovery, verify a new source change in the target. A RUNNING status alone does not prove that PostgreSQL CDC resumed. See the replication-slot lifecycle.

Recovery is refused

capture.recovery-log-gap means retained history no longer contains a change the consumer needs. capture.recovery-progress-unproven means the retained checkpoint cannot prove a safe continuation, including some older stores. v0.7 prevents new unsafe checkpoints but does not repair existing ones.

Back up the metadata and operator state before making changes. Keep the full diagnostic and compare the source, retained position, and target. If you choose restart <pipeline-id> --rerun, review the state-clearing confirmation and target write strategy. A shared chain retained by another consumer can retain completed-table history, so clearing one pipeline does not guarantee a full reload. See Resume or rerun.

Lifecycle, position, and schema sync diagnostics

Control plane and REST API diagnostics

Check target correctness

• Confirm that updates and deletes have a stable key.
• Distinguish append behavior from key-based update behavior.
• Review automatic table creation and schema-change policy.
• Check target constraints, column types, partitioning, and conflict handling.
• Design downstream processing to tolerate duplicate delivery unless your deployment documents a stronger delivery guarantee.

Storage and connector state diagnostics

Interpret observations carefully

• Status separates lifecycle intent from the latest runtime observation. Metrics require an observation and may lag a lifecycle command briefly.
• status --watch and logs --follow continue until you press Ctrl+C.
• Logs are node-local in the preview and are not a durable audit history.
• snapshot can report that no snapshot is available. Confirm source and target records instead of treating that response alone as proof of success or failure.
• Shared-capture inbound observations can be attributed incorrectly in v0.7. Compare per-table target confirmations and data before diagnosing a consumer from inbound counts alone.

Collect a useful diagnostic bundle

Before escalating, collect:

• output from tapstate version and the deployment type;
• connector ID, role, and mode;
• redacted resource files;
• exact command and exit code;
• complete diagnostic or runtime error;
• relevant connector-page URL;
• external-system version and topology;
• timestamp and smallest reproducible dataset;
• expected and observed records.

Remove passwords, tokens, private keys, and customer data before sharing the bundle.

v0.6 diagnostics

Start with the pipeline's Monitoring and Logs pages. For a repeatable diagnostic read, use the observation interfaces.

Engine out of memory

After the engine is lost to out-of-memory, affected running or paused pipelines report engine.out-of-memory and /healthz returns 503 broken. Restart the server process after addressing memory pressure; repeating pipeline start cannot restore the lost engine. Check heap allocation, source batch sizes, operator state, and Nest capacity.

A healthy /healthz response does not prove target writes or CDC continuity. Inspect data and observations after recovery. Pipelines that failed for another reason before a member restart retain that failure and need separate diagnosis.