Mixture of models and semantic routing | Colin McNamara | AIMUG Oct 2026

Colin McNamara explains how semantic routing cuts costs and boosts security by mixing models. Next AIMUG meetup: Mon Nov 2, free (Austin + online): https://aimug.org/events?utm_source=y... For engineers and architects managing LLM costs, privacy, and security in production. What you'll learn Understand that every single request going to the same model is a problem that increases cost and risk. Learn how to route simple requests locally to keep private data from leaving your network. See how a secondary control layer can enforce PII masking and prompt injection protection. Discover that vLLM Semantic Router adds about 240 milliseconds to session initialization. Understand that most requests do not need the biggest model, allowing for efficient resource use. Resources mentioned Slides: https://www.colinmcnamara.com/talks/m... vLLM Semantic Router: https://vllm-sr.ai/ LangChain VLLM NVIDIA Switchyard llm-d OpenTelemetry Docker Kine 00:00 The Single Model Problem 03:55 Mixture of Experts Concept 04:43 Mixture of Models Strategy 07:11 Primary and Secondary Controls 09:35 Efficiency and Token Savings 12:47 The Routing Software Stack 15:42 Protocol Support and Patches 17:42 Key Takeaways and Setup 19:21 Q&A and Latency Details Speaker: Colin McNamara. Slides: https://www.colinmcnamara.com/talks/m.... vLLM Semantic Router: https://vllm-sr.ai/ Slides and notes for all talks from this night: https://aimug.org/docs/oct-2026/?utm_... Newsletter: https://aimug.org/community?utm_sourc... Discord:   / discord   Subscribe to AIMUG on YouTube:    / @ai-mug   Recorded at AIMUG, October 5, 2026. #LLM #SemanticRouting #AIOps