MySQL

Before you begin

Connector availability

The MySQL connector is registered automatically by the Quickstart script. On standalone or fresh server container deployments, the connector catalog starts empty. Confirm that MySQL is registered before authoring connections:

connectors
register mysql

See When registration is required.

Source

Choose the outcome you need. Each tab is a complete preparation path; you do not need to combine steps from both tabs. In tapstate, a connection's mode describes what it can read: snapshot is a bounded full load and cdc is a change-capable source.

Ensure the tapstate runtime can reach the MySQL host and port. Prefer a private network path; when using a routed or internet path, allow only the runtime origin and protect the connection appropriately.

CREATE USER 'tapstate_reader'@'%' IDENTIFIED BY 'your_password';

GRANT SELECT ON `your_database`.* TO 'tapstate_reader'@'%';
FLUSH PRIVILEGES;

CREATE USER 'tapstate_reader'@'%' IDENTIFIED BY 'your_password';

GRANT SELECT ON `your_database`.* TO 'tapstate_reader'@'%';
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'tapstate_reader'@'%';
FLUSH PRIVILEGES;

[mysqld]
server_id         = 223344
log_bin           = mysql-bin
binlog_format     = ROW
binlog_row_image  = FULL

# MySQL 8.0.1 and later: use this retention setting.
binlog_expire_logs_seconds = 604800

# Earlier versions: use this setting instead of binlog_expire_logs_seconds.
# expire_logs_days = 7

SHOW VARIABLES WHERE Variable_name IN (
  'log_bin',
  'binlog_format',
  'binlog_row_image'
);

Create a connection

Tapstate stores the MySQL source as a .tap.yml file in the workspace.

The example below creates a CDC-capable source connection. The connection's mode controls its read capability.

tapstate new --non-interactive \
  --kind source \
  --connector mysql \
  --id orders_source \
  --mode cdc \
  --set host=db.internal \
  --set port=3306 \
  --set database=orders \
  --set username='${MYSQL_USER}' \
  --set password='${MYSQL_PASSWORD}' \
  --workdir tapstate-work

For a full-load-only source, change --mode cdc to --mode snapshot. The generated connection uses the same MySQL fields; only the read mode differs.

Edit tapstate-work/source/orders_source.tap.yml to select tables and the reading behavior:

version: tapstate/v1
kind: source
id: orders_source
connector: mysql
config:
  deploymentMode: standalone
  host: db.internal
  port: 3306
  database: orders
  username: ${MYSQL_USER}
  password: ${MYSQL_PASSWORD}
  addtionalString: "useUnicode=yes&characterEncoding=UTF-8"
  timezone: "+00:00"
mode: cdc
tables:
  - orders
  - /customers_.*/

For a one-time full load, use mode: snapshot. For a CDC-capable source, use mode: cdc after completing the Binlog preparation. Before production, validate the intended full-load and CDC behavior in a staging pipeline.

Use a literal item such as orders for a fixed table. Use a /regex/ item to discover matching tables; keep patterns narrow so a newly created table is not included accidentally. Continue with the MongoDB target guide and pipeline reference to complete the current product path.

Use a primary-replica topology

Set deploymentMode: master-slave, then list the primary first. Use this path instead of standalone host and port.

config:
  deploymentMode: master-slave
  masterSlaveAddress:
    - host: mysql-primary.internal
      port: 3306
    - host: mysql-replica-1.internal
      port: 3306
  database: orders
  username: ${MYSQL_USER}
  password: ${MYSQL_PASSWORD}

For CDC from a replica, make sure it writes replicated events to its own binlog (log_slave_updates = 1). Check SHOW REPLICA STATUS (or SHOW SLAVE STATUS on older MySQL) and resolve replication lag before using it as a source.

Validate the configuration

Run validation after creating or editing the connection files:

tapstate validate --workdir tapstate-work

The validation result shown above illustrates the CLI resource contract. Filenames, locations, resource counts, and error codes vary with the workspace.

Validation does not require all connection fields, reject unknown connector keys, resolve secrets, or contact MySQL. A syntactically valid resource that uses mysql.invalid can therefore pass validation.

To check the connection, register the MySQL connector JAR, apply the workspace, and run the test from an authenticated REPL:

tapstate -w tapstate-work

connect http://127.0.0.1:8080
login admin
apply
test orders_source -o json

Only outcome: PASSED establishes that the connector accepted the supplied endpoint and credentials. A placeholder or unreachable host should produce outcome: FAILED or a coded connector test error; the exact checks and timeout come from the registered connector. See Register and test connectors for the full sequence.

Limitations

Source limitations

• Only tables and indexes are in scope. Temporary tables, hidden columns, triggers, views, and stored procedures are not replicated.
• CDC parses the MySQL binlog and can increase source CPU and disk I/O under a high change rate. Monitor both resources during peak workloads.
• Keep Binlog files for at least seven days, and longer than the longest realistic outage or maintenance window. A CDC job cannot start or resume from a position that has expired or been deleted.
• Set binlog_format = ROW and binlog_row_image = FULL. Ensure binlog-do-db includes the selected database and binlog-ignore-db does not exclude it.
• Primary-replica addressing requires a planned failover test. Before a switchover, verify replication consistency, log_slave_updates = 1 on the replica used for CDC, and acceptable replica lag.
• In MySQL 5.6 and earlier, normalize negative TIME values before CDC testing.
• MySQL-compatible databases can add non-native schemas or functions that the MySQL parser does not understand. Test those variants before production use.

Consistency and conflicts

• Writes made outside the replication path can cause source-target differences.
• Avoid DDL on participating source tables during a snapshot. Schema changes can invalidate the snapshot or cause it to fail.
• Validate row counts and keyed samples after a snapshot, then test INSERT, UPDATE, and DELETE events before opening the target to consumers.

Reference

Connection settings

Data compatibility

• Current MySQL → MongoDB path: TIMESTAMP(3) → BSON Date at millisecond precision. Preserves the instant; does not preserve the source time-zone or offset representation.
• Replicated row changes include INSERT, UPDATE, and DELETE.
• Adding, changing, renaming, and dropping columns remain pipeline-level staging checks.
• Test heterogeneous pipelines with spatial, custom, ENUM, and SET values because downstream systems commonly represent them as strings.