What Is Fine-Tuning? Explained For Software Testers | AI Tester Dictionary #028

Imagine you are building an AI system that classifies customer support requests for one particular industry. A general purpose model already handles the simple cases. But your organisation has its own terminology, its own classification system and its own very specific rules, and you have hundreds of thousands of historical requests showing exactly how your team classified each one. You want the model to follow those patterns every time, without explaining all of them in every prompt. Fine-tuning means taking a model that already exists and training it further on those examples. What comes out is the same model, except that its learned parameters have been adapted to your job. Building one from scratch is a different exercise entirely, and it takes data and compute on a scale almost nobody has. Then there is the part that changes your job as a tester. A model can score perfectly on its own training examples and still fail completely on anything else. Your data has three ways of saying a customer was charged twice, all labelled billing dispute; production sends a fourth nobody wrote down. If that comes back as anything else, the model learned your examples rather than your task. And there is a cost on the other side. Afterwards the model is better at the task you trained it for, but its other abilities have moved, because the parameters that were carrying them are the same ones you just retrained. This has a name: catastrophic forgetting. The episode works through the six-step loop and why the split matters most, how fine-tuning differs from prompt engineering and from RAG, and the four things to test on a fine-tuned model. ⏱️ Chapters 0:00 One job, done your way 0:32 What the AI Tester Dictionary is 0:42 A model, plus your examples 1:13 Six steps, and a loop 1:38 What the data looks like 2:14 Instructions or parameters 2:42 Two different problems 3:11 Testing on what it never saw 3:45 What the fine-tune cost 4:13 Four things to check 4:43 What to hold on to 🔑 Key takeaways • Fine-tuning takes a pre-trained model and trains it further on your examples, so what changes is the model itself — not the instructions in front of it • Prompt engineering is cheaper and iterates in seconds, and RAG is for information that moves. Both are worth exhausting before you pay for a training run • A score on the training examples tells you nothing. The number that means something comes from data the model has never seen, labelled before you ran it • Fine-tuning moves parameters that were carrying other behaviour, so a fine-tuned model is a new release and needs the whole regression suite, not just the new task's tests 📚 In this series #001 — Artificial Intelligence #002 — Machine Learning #003 — Deep Learning #004 — Generative AI #005 — Large Language Models #006 — Transformer #007 — Diffusion Model #008 — Foundation Model #009 — Token #010 — Context Window #011 — Prompt #012 — System Prompt #013 — Prompt Engineering #014 — Temperature #015 — Top K and Top P #016 — Hallucination #017 — Embeddings #018 — Vector Database #019 — Semantic Search #020 — Retrieval-Augmented Generation #021 — Chunking #022 — Reranking #023 — Knowledge Graph #024 — Multimodal AI #025 — Function Calling #026 — Structured Output #027 — Grounding #028 — Fine-Tuning ← you are here ▶️ Previously #027 — Grounding: tying what a model says to evidence it was actually given, and the question that changes your testing.    • What Is Grounding? Explained For Software ...   ▶️ Next in the series #029 — Context Engineering: what you put in front of the model in the first place. 🔔 Subscribe for practical AI and software testing content. #SoftwareTesting #AIForTesting #AITesterDictionary #FineTuning #LLM #MachineLearning #QAEngineer #SDET #AITesting #TestAutomation #ModelEvaluation #QualityEngineering #SoftwareQuality FOLLOW ME 📸 Web: https://softwaretestingtrends.com/ Discord:   / discord   LinkedIn:   / ambysan   Twitter:   / ambysan   Facebook:   / softwaretestingtrend   Medium:   / ambysan   BLOGGING 📝 Software Testing Trends: https://softwaretestingtrends.com/ TechJournal: https://ambreenkhan.dev/ ABOUT ME 👩‍💻 https://ambreenkhan.netlify.app/