Day 5 · A grader you can trust | FDE First Principles

What exactly makes a reply 'correct' for this customer? Day 5 of 66 in FDE First Principles · Project 1 of 6: Measure before you believe A public learning log: a software architect learning AI from first principles for the Forward Deployed Engineer role. One topic at a time, built from scratch and explained in plain words. In this video ▸ For a beginner: Code-based asserts give basic confidence in evals, and the code itself also needs to be verified. The limits of code-based asserts, and what other evals could be used, is the next thing to learn. ▸ For an engineer: score() checks that the assert conditions are met for a test case, for one aspect of validation. A total score may carry a weighting of the individual eval assertions; in this case it is simple — each type of eval must pass. ▸ For a customer: The eval checks all aspects of the response before claiming a pass, because an individual check may look good while the response in the grand scheme of things may still fail. Learn it yourself (free) • Define success criteria and build evaluations (Claude docs): https://platform.claude.com/docs/en/t... My code and journal for this topic https://github.com/rajchampaneriya/co... The whole learning log: https://github.com/rajchampaneriya/co... Previous: Day 4 · Version A: rules and templates (the honest baseline):    • Day 4 · Version A: rules and templates (th...   How this series runs Every topic drops on a schedule with a deadline, a calendar event and this video as its finish line; each project ends with an article. The coach that runs it is open source: FDE Impromptu Coach, https://github.com/rajchampaneriya/fd... Articles: https://rajc.work #FirstPrinciples #AIEngineering #ForwardDeployedEngineer