AI Engineering Concepts Every DevOps Engineer Must Know | DevOps Daily Dose Podcast

It's 2 AM, you get paged, and the new workload isn't a 500 MB container — it's a model on a rack of GPUs that answers the same request differently every time and logs its hallucinations as 200 OK. This deep-dive audio podcast builds a complete mental model of AI engineering for DevOps: tokens and tokenizers (BPE, Unigram, SentencePiece), context windows and context rot, prompts as code, prompt caching and the KV cache, fine-tuning with SFT, DPO, RFT and LoRA, RAG with embeddings, vector and hybrid search and agentic RAG, tool calling and the Model Context Protocol (MCP), GPUs on Kubernetes with the NVIDIA GPU Operator, serving with NVIDIA NIM, KServe and vLLM PagedAttention, LLMOps evals and latency optimization — all the way to the OWASP Top 10 for LLM Applications, prompt injection, excessive agency and the NIST AI Risk Management Framework. Who it's for: DevOps, SRE and platform engineers, cloud and infrastructure engineers and students — beginners through interview level — who now have to deploy, scale, observe and secure LLM workloads and want to understand AI engineering end to end in the infrastructure terms they already know. Chapters: 00:00 Intro 00:17 The 2 AM page: why AI breaks deterministic infrastructure 03:37 Tokens, transformers & self-attention 07:13 Tokenizers: BPE, Unigram & SentencePiece 12:08 Context windows, token accumulation & context rot 15:54 Prompts as code, prompt caching & the KV cache 22:03 Fine-tuning: SFT, DPO, RFT & LoRA adapters 27:20 RAG: embeddings, vector search & retrieval 30:32 Hybrid search, re-rankers & agentic RAG 34:21 Tool calling & agents: client vs server tools 38:28 Model Context Protocol (MCP): hosts, clients & servers 42:01 GPUs on Kubernetes: NVIDIA GPU Operator, GFD & DCGM 44:32 Serving: NVIDIA NIM, KServe, vLLM & PagedAttention 49:37 LLMOps: evals, LLM-as-a-judge & latency optimization 53:53 Security: OWASP LLM Top 10, prompt injection, excessive agency & NIST AI RMF 59:17 Mental-model recap & the future of AI SREs 62:09 Outro Built from official documentation: OpenAI, Anthropic, Google (Machine Learning Crash Course & Google Cloud), Microsoft Azure, AWS, NVIDIA (Triton, NIM, GPU Operator), Hugging Face, Kubernetes, KServe, vLLM, Kubeflow, MLflow, the Model Context Protocol specification, OpenTelemetry GenAI semantic conventions, OWASP GenAI Security Project, NIST and GitHub Docs. 🎧 Plug in your headphones and listen & learn. If this helped, subscribe to @DevOpsDailyDose and hit the bell for more deep dives! #AIEngineering #DevOps #LLMOps #MLOps #Podcast