4.3 · The Retry That Refunded Twice (Retries, Idempotency and Evaluator Loops)

A payment API timed out on Priya's $120 refund, so our code retried, and now she has been refunded twice. Find out why: the first call went through and only the reply was lost, because a timeout doesn't mean it failed. Then learn which errors are worth retrying (timeouts, 429s, temporary 5xx) and which aren't (an amount larger than the order), capped exponential backoff, full jitter, why retries belong in one place in the stack, and the real fix: an idempotency key that the server remembers, with Stripe as one real example. Test yourself with a fresh key on every attempt, see where a good key comes from, and finish with the evaluator-optimizer loop: a writer and a critic that revise a reply, and what to do when the critic never passes. Lesson 4.3 of the AI Agents course. The payment API is simulated (FakeRefundAPI), timeouts and backoff sleeps are recorded, not slept, the critic is a string check standing in for an LLM evaluator, and Nimbus, Priya, the order and every amount are demo data from the course code. Chapters: 0:00 The retry that refunded twice 1:47 Which errors are worth retrying 2:43 Backoff, jitter and one retry layer 4:08 The real fix: idempotency keys 5:48 Quiz: a new key on every attempt 6:38 Where the key comes from 7:34 Evaluator loops: repeat on purpose 8:27 When the critic never passes Mid-video quiz: the provider now times out after committing twice (FakeRefundAPI(timeouts_after_commit=2)), and the code makes a fresh random key (uuid4) on every attempt, with retry(..., attempts=4). How many refunds does the provider commit? A) 1, B) 2, C) 3, D) 4. The answer is in the video. Sources: AWS, Amazon Builders' Library, Timeouts, retries, and backoff with jitter (Dec 5, 2019): https://builder.aws.com/content/3Eumj... AWS Architecture Blog, Exponential Backoff And Jitter (Marc Brooker, 2015): https://aws.amazon.com/blogs/architec... AWS backoff simulator code: https://raw.githubusercontent.com/aws... Stripe API reference, Idempotent requests: https://docs.stripe.com/api/idempoten... Anthropic, Building effective agents: https://www.anthropic.com/research/bu... LangChain, LangGraph Interrupts: https://docs.langchain.com/oss/python... Instagram: @whiteboardbytes #aiagents #llm #python