Перейти к содержимому

RAC Interconnect Bottleneck — When gc buffer busy Steals Your CPU | Diagnose Cache Fusion Stalls

DBNexus Training & Consulting

0:00 / 0:00

RAC Interconnect Bottleneck — When gc buffer busy Steals Your CPU | Diagnose Cache Fusion Stalls

296 просмотров · 3 недели назад
DBNexus Training & Consulting
19 тыс. подписчиков
296 просмотров · 3 недели назад
Thursday 9:15 AM. Two days after the SAP go-live weekend. Order entry that ran in 80 ms now takes 240. Every screen 3x slower. No ORA error. No node eviction. Both nodes are up. The AWR report has one villain — gc buffer busy acquire, 55% of database time. One index leaf block is pinging between node 1 and node 2 thousands of times a second, over a 1GbE interconnect the go-live team never upgraded. This episode covers the 4 root causes of RAC interconnect stall, the 3 queries that expose the hot block, and the safe interconnect setup that makes gc buffer busy disappear. TIMESTAMPS 0:00 The Thursday-morning go-live brownout 0:07 The true story — 55% of DB time on one wait 1:17 5 things you'll walk away with 1:58 What gc buffer busy actually means (acquire vs release) 2:32 Discovery — 3 queries that expose the interconnect stall 3:20 Cause 1 — Hot block contention (right-hand index) — 55% 4:06 Cause 2 — Interconnect saturation (undersized network) 4:55 Cause 3 — Undo header contention across instances 5:43 Cause 4 — SCN broadcast overhead (high commit rate) 6:28 Bonus — the safe RAC interconnect setup 7:28 STOP — 3 anti-patterns coming up 7:37 Anti-Pattern 1 — Blindly enlarge the buffer cache 8:35 Anti-Pattern 2 — Toggle DRM in a panic 9:20 Anti-Pattern 3 — Ignore interconnect / OS network stats 10:05 The Playbook — 5-minute interconnect diagnosis 10:57 Case file every senior DBA brings 12:08 War Story — the go-live brownout that hid on the wire 14:38 Before you leave 15:35 Upcoming episode — ORA-00060 Deadlock Storms THE 3 DIAGNOSTIC QUERIES 1) SELECT inst_id, event, count(*) FROM gv$session WHERE event LIKE 'gc buffer busy%' GROUP BY inst_id, event; 2) SELECT owner, object_name, class, SUM(cr_block+current_block) xfer FROM gv$instance_cache_transfer JOIN dba_objects USING(data_object_id) GROUP BY owner, object_name, class ORDER BY xfer DESC; 3) SELECT inst_id, name, value FROM gv$sysstat WHERE name IN ('gc blocks lost','gc current block receive time'); THE SAFE RAC INTERCONNECT SETUP Dedicated private 10GbE (or faster) NIC — never shared with public traffic Jumbo frames — MTU 9000 end-to-end, including the switch Let HAIP own the interconnect _gc_policy_time — raise the DRM interval so the cluster stops remastering constantly _gc_undo_affinity — keep undo mastering on the writing node Alert on gc blocks lost greater than 0 and NIC drops, not just Oracle waits THE 4 ROOT CAUSES 1) Hot block contention — monotonic sequence index leaf block pinging across nodes (55%) 2) Interconnect saturation — 1GbE / shared switch / no jumbo frames / packet loss 3) Undo header contention — cross-node consistent reads on hot undo segments 4) SCN broadcast overhead — commit-after-every-row floods the interconnect REFERENCES (MOS Doc IDs) 1445608.1 — gc buffer busy diagnosis in RAC 1912186.1 — RAC commit and SCN broadcast tuning 1367128.1 — Interconnect / private network best practice 759557.1 — Dynamic Resource Mastering (DRM) overview SUBSCRIBE for the next upload. LIKE to help YouTube surface this to other senior DBAs. COMMENT your worst RAC war story. support@dbnexus.co.in +91 8169158909 🔥 JOIN OUR EXCLUSIVE MEMBERSHIP PROGRAM TODAY — only INR ₹1199 Become a member now and enjoy loyalty badges, exclusive members-only videos, early access to new content, priority comment replies, and exclusive DBA content across various topics. ▶ Join:    / @oracledbaonlinetraining   🔥 MEMBERSHIP BENEFITS 1) Loyalty badges 2) Custom emojis 3) Members-only exclusive videos 4) Early access to new videos 5) Priority comment replies 6) Exclusive DBA content on various topics 🎓 GET THE COMPLETE DBNEXUS COURSE ▸ Recorded Oracle 26ai course: https://www.dbnexus.co.in/recorded-26... ▸ Recorded 19c Ultimate course: https://www.dbnexus.co.in/recorded-ul... ▸ Oracle DBA — All YouTube Video Documents: https://www.dbnexus.co.in/recorded-yo... ▸ LIVE courses (weekend / weekday batches): https://www.dbnexus.co.in/courses.html ▸ DBA support & consulting: https://www.dbnexus.co.in/dba-support... ▸ Website: https://www.dbnexus.co.in ▸ WhatsApp: +91 8169158909 ▸ Email: support@dbnexus.co.in #OracleDBA #RAC #gcbufferbusy #CacheFusion #Interconnect #Oracle26ai #DBA #DBNexus 📺 CONTINUE THE SERIES ▸ Previous episode —    • ORA-04030 PGA Exhaustion — The Silent OOM ...   ▸ Two episodes back —    • Adaptive Cursor Sharing Bloat — When One S...   Full playlist (all episodes): →    • Oracle DBA Troubleshooting Masterclass | A...