Sync Monitoring

Backfill

Re-send a table's existing rows when one side is missing history the triggers never captured.

Sync captures changes as they happen. Rows that were already sitting in a table before its triggers existed have never been captured, so they were never sent — the two sides can hold different data while sync itself is working perfectly.

A backfill closes that gap. It walks the source table from top to bottom and stages every row for review. Nothing reaches the receiving side until you have read the staged rows and approved them.

To repair a single row — typically one whose change was skipped in the queue — use Backfill this row on the Queue tab instead. It stages just that row, through the same review, and shows as row … only against the table below.

When you need one

  • After first configuring a table. Everything already in it predates the triggers.
  • After reactivating a table. Changes captured while it was inactive were dropped, not held.
  • After re-running the setup SQL following an outage, if the triggers were absent for a while.
  • When the row counts disagree and the difference is not explained by a recent import still in flight.

Row counts on the school's Table Monitoring panel are the usual signal. They refresh on a slower clock than the sync itself, so a gap immediately after a large import is normal and closes on its own. A gap that persists across several cycles is the case for a backfill.

Before you request one

Three conditions have to hold, and the button stays disabled until they do.

ConditionWhy
The table is activeAn inactive table is not in the configuration the agent fetches
Create and update are both enabledRows are replayed as upserts, which may insert or overwrite
The source side reports more rows than the receiving sideThe button is for a side that is behind; it does not run when the counts already agree

If a request is already open for that table and direction, the button shows its status instead and will not start a second one.

Requesting one

Open the school and find the table on the Table Monitoring panel. Each row carries a Backfill button; the row expands to show operations, apply order and when each side was last counted, which is worth checking before you commit.

Requesting one records it and returns immediately — nothing happens in the dashboard itself. The work is done by the agent on the source side, on its next cycle.

Reviewing the staged rows

A finished replay leaves the request AWAITING REVIEW and the button becomes Review, with the number of rows waiting. Opening it lists every staged row: the primary key, the first few columns inline, and the whole captured payload on hover. Long replays page in as you scroll to the end of the list.

Two outcomes:

  • Approve and release copies the staged rows into the queue, where the receiving agent pulls them like any other change. Large replays release in the background; the request shows RELEASING until the last row is through.
  • Reject closes the request without sending anything. The staged rows are kept, so what was turned down stays readable until the retention sweep removes the request.

Either way the request is recorded against your account with the time you reviewed it, and a rejection can carry a note.

Staged rows are not queued rows

Until they are approved, staged rows live apart from the sync queue. The receiving agent cannot see them, they do not appear in sync history, and they count for nothing in the queue lag figures.

What the agent does

The backfill step runs at the end of a cycle, after the agent has pushed the changes it captured normally. This side's live changes are never held up by a replay.

Rows are paged by primary key, not by offset — each page asks for rows after the last key it saw. A table being written to during the walk cannot shift rows across a page boundary and have them skipped.

Rows are sent as upserts. The receiving side inserts what it does not have and overwrites what it does, rather than reporting every existing row as a conflict. This is why create and update must both be enabled.

A resume point is used where one is known. If the receiving side's highest key is already past part of the table, the walk starts from there instead of replaying rows that side demonstrably holds.

From the moment they are approved, backfilled rows are ordinary queued changes. They appear in sync history, respect apply order, and are acknowledged like anything else.

Reading the result

The button reflects the request's state until it finishes.

StateMeaning
BackfillNothing outstanding for this table and direction
PENDINGRecorded, waiting for the source agent's next cycle
IN PROGRESSAn agent has claimed it and is walking the table
Review nThe walk finished; n rows are staged and waiting on you
RELEASINGApproved, and the staged rows are being copied into the queue
button returns to BackfillFinished or rejected — the request is closed and counted in history

A finished request records how many rows it staged and how many it released. A failed one records why. All of them age out with the retention sweep once they pass the history window; requests still in flight are never removed.

The row counts are the real confirmation. They refresh on their own slower clock, so give them a cycle or two to catch up before concluding anything went wrong.

What a backfill does not do

A backfill only sends rows the source has. It never removes rows the receiving side is holding but the source no longer has.

That matters where a table has been syncing for a while with deletes disabled. Rows deleted on the source were never propagated, so the receiving side still holds them — and a backfill will not clear them, because it only ever pushes what currently exists. Enabling Delete rows on the table stops the gap growing from that point on; it does not undo what accumulated before. Reconciling the leftovers is a separate job.

Counts can still differ after a successful backfill

If the receiving side ends up with more rows than the source, the surplus is this — rows whose deletion never travelled. A backfill cannot bring that number down.

Limitations

A large table is walked across several cycles

Each cycle replays up to a fixed number of rows and then hands the request back, so the agent's own capture pass is never held up for long. The request stays open, showing the rows staged so far, until the walk reaches the end of the table.

An abandoned walk is picked up again

If the agent stops mid-walk — a crash, a killed process, a lost connection — the request stops being renewed and another agent claims it once the lease expires, resuming from the last key it confirmed. Rows staged by the abandoned attempt are not sent twice.

On this page