One LLM, Many Users 🦀 Rust Mutex, RwLock & a Real Deadlock | Concurrency Part 2

🦀 Sixteen users, one LLM engine in plain Rust, eight worker threads. One worker served all sixteen requests while the other seven waited, because of a single `while let`. In this video the engine becomes a server with shared state, we hit the classic shared-state bugs for real, and we measure every fix on an Apple M2 Pro. 1. A request queue and token stats behind Mutex: `while let` keeps the MutexGuard alive for the whole loop body. 16 requests in 8.39 s on one worker → 1.17 s on all eight, 7.2× faster, by dropping the guard at the semicolon. 2. Poisoning: one 0-token request panics under the stats lock, and 8 PoisonErrors take the server down. The fix: no panics under a lock, and PoisonError::into_inner when the data is still valid. 3. A hot model reload with RwLock: the writer waits 286 ms for eight readers, and new readers queue behind it. Copy the Arc out of the read guard: the reload takes 83 ns, and eight stories finish on the old weights. 4. A prompt cache with a check-then-act race: 11 stories written for 2 prompts. One OnceLock per prompt: 2 stories, 2.1× faster. 5. A real deadlock between two locks, caught by a watchdog with JoinHandle::is_finished and try_lock, fixed with one lock order. And one thread that deadlocks itself: a read guard still alive when it asks for the write lock. Rust rules out data races at compile time, but race conditions, starvation and deadlocks all compile. Every number on screen comes from a real run at that step of the code. 💻 Full source code, with a git tag for every step: https://github.com/edgflow/rs-fast-llm ✅ What you'll learn: • Arc vs Mutex: why shared data needs a lock to change, and why the data lives inside the Mutex • MutexGuard: what lock() returns, Deref, and unlocking on drop • Where a guard is dropped: while let vs a let statement, and lock starvation • Mutex poisoning, PoisonError and into_inner • RwLock: many readers or one writer, writer waits, and a hot reload in nanoseconds • Race conditions vs data races: check-then-act, fixed with OnceLock • Deadlocks: the four conditions, a wait-for cycle, lock ordering, try_lock and a watchdog • The one-thread RwLock deadlock (read guard + write lock) ⏱️ Chapters: 0:00 Intro: one worker did all the work 0:54 From part 1 to an LLM server 1:41 A server: Arc and Mutex 2:36 Memory graph: inside a Mutex 3:38 Worker threads and a request queue 4:28 Starvation: while let holds the lock 5:27 The fix: drop the guard at the semicolon 6:00 A panic poisons the lock 7:00 Recovering with into_inner 7:48 RwLock and a hot model reload 8:55 Why the reload waited 286 ms 9:31 Copy the Arc out: a reload in 83 ns 10:18 A prompt cache: check-then-act race 11:35 OnceLock: every story written once 12:34 A real deadlock and a watchdog 14:01 Four conditions and one lock order 15:02 One thread, one RwLock: stuck 15:54 Tests 16:20 Recap and part 3 📺 Part 2 of 6: Rust Concurrency for AI. Part 1: Rust Threads for AI. Next: atomics and memory ordering. 🧰 Rust 1.94 (edition 2024) · std::sync (Arc, Mutex, RwLock, OnceLock) · clap 🎙️ The narration voice is AI-generated (Google AI Studio). 💬 Which of these bugs have you met in your own code? Tell me in the comments! 🔔 Subscribe to @aiedgeflow for more Rust + AI. #rust #rustlang #ai #llm #concurrency #mutex