Partitioned Consumer Groups in NATS: Ordered, Scalable Stream Processing

Need messages for the same customer processed in order — but still want to scale across a fleet of workers? Plain load balancing splits related messages across workers and breaks the ordering you care about. This video shows how NATS solves that with partitioned consumer groups: deterministic partitioning routes every message with the same key to one worker, while thousands of keys run in parallel. We build it up from two JetStream primitives, walk static vs. elastic groups, then prove it out with live CLI demos and a Go app. What you'll learn: Why ordinary load balancing breaks per-key ordering — and how partitioning fixes it The two building blocks: deterministic subject-transform partitioning + pull-consumer pinned client policy max_ack_pending=1 as the ordering safeguard, plus failover with hot standbys Static vs. elastic groups: subject transforms up front vs. a hidden work queue stream with live add/drop Handling hot partitions with custom member mapping Warm-cache routing: key affinity without strict serialization Live CLI walkthrough with the CG tool, plus a Go + SSE visualization demo -- Timestamps -- 0:00 Why per-key ordering breaks under load balancing 0:31 The goal: order per key, parallel across keys 0:58 How it works: partitioning + priority groups 4:07 Building it up from a basic stream (Alice & Bob) 6:32 max_ack_pending=1: server-enforced ordering 8:12 Pinned client policy and failover 8:37 Vocabulary: stream, group, partition, member 10:24 Static vs. elastic group diagrams 12:01 Hot partitions and custom mapping 13:11 Static limits, and what elastic lifts 14:56 Warm-cache routing without serialization 16:11 Static vs. elastic: which to reach for 17:08 CLI demo: static group with the CG tool 20:20 CLI demo: elastic group, live add/drop 25:48 Go code walkthrough 28:02 Go demo: static page 28:49 Go demo: elastic page, add & drop members live 30:29 Wrap-up -- Resources -- NATS docs: https://docs.nats.io NATS GitHub: https://github.com/nats-io Synadia: https://www.synadia.com Partitioned Consumer Groups Orbit: https://nats.io/blog/orbit-partitione... #NATS #JetStream #Messaging #CloudNative #GoLang #StreamProcessing #ConsumerGroups