AI Observability for Local RAG Apps | Phoenix UI Evaluation Dashboard

In this video, I show how to add AI observability and evaluation to a local RAG app using Phoenix. When building a RAG application, it is not enough to only check the final answer. You also need to understand what happens behind the scenes: which chunks were retrieved, how close the results were, whether retrieval quality improved, and whether unsafe prompts were blocked. In this demo, I open Phoenix while asking questions in the app and show the RAG pipeline live. Phoenix UI: http://localhost:6006 Collector endpoint: http://localhost:6006/v1/trace In this video, I cover: How Phoenix helps monitor a RAG app How to view traces from your local AI app How to inspect retrieved chunks How to compare chunk distances How to use observability to improve retrieval quality How to test prompt injection attempts How unsafe prompts can show up clearly in the dashboard Why observability matters before moving an AI app closer to production Demo scenarios: Live RAG Health Dashboard I ask questions and show how Phoenix captures the retrieval flow, chunk selection, and distance scores. Retrieval Improvement Demo I show how better ingestion, chunking, or embeddings can improve the retrieved context. Security Demo I send a prompt injection attempt like “ignore previous instructions,” then show how the guardrail blocks it and how the trace appears in Phoenix. This is a great portfolio demo because it proves you are not only building an AI app, but also thinking about quality, debugging, safety, and production readiness. By the end of this video, you will understand how Phoenix can help you monitor, evaluate, and debug a local RAG app with real traces and live AI observability. #AIObservability #PhoenixUI #RAG #LocalAI #AIEvaluation