The AI Can't Run Your Tests. It Asks for a Tool. | Tool Calling

Ask an AI model about your test results, and it can write a sentence like "I found three failing tests". Ask it to actually run the tests, and it cannot open your test runner by itself, because all it can do is produce text. It needs a way to reach an external system, and the usual way is called tool calling. Tool calling lets an AI model ask for a predefined tool to be run. The model does not run the tool itself. It writes a structured request that says which tool it wants and which arguments the tool should receive, and the application around the model runs the tool and sends the result back. So on every call there are two things to check: the request the model made, and the result it got back. A common belief is that tool calling lets an AI use any tool it wants. In practice, the application decides which tools exist, what arguments each one accepts and what each one is allowed to do. The model can only ask for a tool on that list, so the list is a security control, and it is worth testing like one. The episode also follows a testing agent through a checkout investigation in three tool calls, drives a login check through browser tools, and covers four checks on every call, what to do when a tool fails, and how a tool differs from an API. ⏱️ Chapters 0:00 Answering, or acting 0:26 What the AI Tester Dictionary is 0:36 The model asks, the application runs 1:03 A controlled connection 1:34 One investigation, three calls 2:13 Every step is a tool call 2:51 Four checks on every call 3:20 A failing tool is a test case 3:49 Only the tools you provide 4:20 A tool often wraps an API 4:51 What to hold on to 🔑 Key takeaways • Tool calling is a loop: the model asks for a predefined tool, the application runs it, and the result comes back for the model's next decision • The model never runs a tool itself, so check every call twice: the request it made, and the result it got back • On every call, check that the model chose the right tool, sent the right arguments, read the result correctly, and was allowed to take the action • Make a tool fail on purpose and watch what the agent does, and log every tool call so you can see afterwards what it asked for and what came back • The application decides which tools exist and what they may do, so the tool list is part of your security: try to get the agent to use a tool it should not have 📚 In this series #001 — Artificial Intelligence #002 — Machine Learning #003 — Deep Learning #004 — Generative AI #005 — Large Language Models #006 — Transformer #007 — Diffusion Model #008 — Foundation Model #009 — Token #010 — Context Window #011 — Prompt #012 — System Prompt #013 — Prompt Engineering #014 — Temperature #015 — Top K and Top P #016 — Hallucination #017 — Embeddings #018 — Vector Database #019 — Semantic Search #020 — Retrieval-Augmented Generation #021 — Chunking #022 — Reranking #023 — Knowledge Graph #024 — Multimodal AI #025 — Function Calling #026 — Structured Output #027 — Grounding #028 — Fine-Tuning #029 — Context Engineering #030 — AI Agents #031 — Tool Calling ← you are here ▶️ Previously #030 — AI Agents: a model plus the system around it, working in a loop towards a goal.    • This is what separates an AI Agent from an...   ▶️ Next in the series #032 — Agentic AI: systems that work towards a goal with less step-by-step direction. 🔔 Subscribe for practical AI and software testing content. #SoftwareTesting #AIForTesting #AITesterDictionary #ToolCalling #AIAgents #FunctionCalling #QAEngineer #SDET #AITesting #TestAutomation #Playwright #QualityEngineering #SoftwareQuality FOLLOW ME 📸 Web: https://softwaretestingtrends.com/ Discord:   / discord   LinkedIn:   / ambysan   Twitter:   / ambysan   Facebook:   / softwaretestingtrend   Medium:   / ambysan   BLOGGING 📝 Software Testing Trends: https://softwaretestingtrends.com/ TechJournal: https://ambreenkhan.dev/ ABOUT ME 👩‍💻 https://ambreenkhan.netlify.app/