Plan nest capacity and delivery behavior
Size a nest pipeline around whole-document writes, final-state delivery, and representative source changes
nest writes an assembled document as a unit. A change to a row that belongs
to a document can therefore rewrite that complete document, not only the
changed row. Plan and test this path using the largest document and busiest root
you expect, not only average rows.
Understand the delivery contract
The current preview uses a 50 ms leading-edge send window for a normal nest
root. The first changed document is sent immediately; later changes during that
window can be folded into the next document write. A quiet document does not
wait for the window.
A deletion is sent immediately and closes the open window. It is not folded with later changes because the document key may no longer exist.
This is final-state delivery, not an audit stream of every intermediate
version. It fits an upserting target such as the published MongoDB path. If a
consumer must receive every version, a normal folded nest path is not an
appropriate contract. root.mode: append avoids folding, but it changes the
target semantics and must be evaluated with representative data.
The 50 ms window is preview behavior, not a latency or throughput service-level objective. Do not size a production deployment from this value.
Plan with four inputs
Before a realistic test, record:
- the largest expected assembled document size;
- the maximum number of embedded elements under one root;
- the peak change rate for one busy root and the aggregate source change rate;
- target write capacity, indexes, and write concern for the actual topology.
Whole-document writes make both document size and fan-out important. A source table can have modest row sizes while its assembled document is too large for the target or too expensive to rewrite at its peak update rate.
Test the target behavior
Use a non-production environment that matches the intended schema and indexes. Test at least:
- an initial load with documents near the largest expected size;
- a sustained update run against the busiest root;
- concurrent updates across many roots;
- inserts, updates, deletes, and structural key changes where enabled; and
- target reads that confirm consumers see the expected final document.
Watch metrics, logs, target write latency, source retention, CPU, memory,
and storage. A running pipeline is not enough evidence: verify target data
and investigate any dead-lettered records or repeated failures.
Keep the document model bounded
Set an explicit upper bound on business fan-out before authoring a deeply nested tree. When an assembled document can grow without a practical bound, split the consumer model or use a different materialization shape. Increasing a memory budget cannot make an oversized document safe to assemble or store.
The preview is single-node and does not publish a production throughput SLO. Treat measured capacity as deployment-specific and repeat these tests after changing the source schema, document shape, target indexes, or server version.