Skip to content
Syncline
advanced
10:04

Retries, rate limits and a safe backfill

What happens after a run goes wrong, and what happens when you point a flow at eleven thousand old rows. Which errors get retried and which never will, how a queue looks different from a failure, and the backfill settings worth checking twice.

Video placeholder

Three stacked panels: a retry timeline with four attempts fading out, a paused queue badge in amber, and a backfill progress bar reading eleven thousand records with a batch size field beside it.

Chapters

<ul class="sy-chapters"><li><b>0:00</b> Temporary errors versus permanent ones</li><li><b>1:42</b> The four attempt retry window</li><li><b>3:35</b> Queued is not failed: reading the paused badge</li><li><b>6:10</b> Setting up a backfill against a test destination</li><li><b>8:20</b> Batch size, duration and what it costs</li></ul>

From the transcript

Three things here, and the last one is where people get hurt. Retries first. When a destination returns a temporary error we back off and try again: four attempts over about fifteen minutes, then the run goes to failed and sits there waiting for you to replay it. A permanent error, like a field that does not exist, is not retried at all, because trying again will fail in exactly the same way. Rate limits next. Every app has one, and ours sits separately from theirs. When we get close we queue rather than drop, and you see a paused badge instead of a red one, which is a very different thing to panic about. Now the backfill. You have eleven thousand old rows and you want them all in the other app. Do not switch on a normal flow and walk away. Use the backfill runner, set a batch size, and point it at a test destination first. Eleven thousand records at five per second is roughly forty minutes, and every one counts against your monthly tasks.

Apps in this video

airtable, gmail, slack

More videos

Retries, rate limits and a safe backfill

What happens after a run goes wrong, and what happens when you point a flow at eleven thousand old rows. Which errors get retried and which never will, how a queue looks different from a failure, and the backfill settings worth checking twice.

Chain Stripe, HubSpot and Trello in one flow

One trigger, three actions, and the order decides whether the result is usable. A first payment creates the contact, moves the deal and opens the onboarding card, each step handing its output to the next, with a clean stop when the first one fails.

Read a run history and fix a broken mapping

The flow is rarely broken. One record is failing over and over, and the history names it in about thirty seconds. Sort by failed, read the two columns beside the error rather than the error itself, fix the mapping, replay the backlog in one pass.