What Is Multimodal AI? Explained For Software Testers | AI Tester Dictionary #024

A test fails and leaves you four kinds of evidence: an error line, a screenshot, a recording of the run, and a voice note from the person watching. You paste the error line into the assistant, because text is the only thing it can read. Everything else you look at yourself, then type up what you saw. A modality is a kind of information — text, image, audio, video. A model that only takes text can only read what you type. A multimodal model takes more than one kind, often in the same request, so the screenshot and your question end up in one place where the model can reason about both. For a tester that is the difference between asking about your notes on a screen recording and asking about the recording. It is also where a new failure shows up. The model can read the text correctly, read the image correctly, and still connect them wrongly, for example by matching the error to the wrong part of the screenshot. Each input looks fine, so nothing flags the wrong conclusion. The episode works through a real defect assembled from three attachments, the three levels worth testing when most teams test one (and a quick way to test the bottom two), and two things people assume: that multimodal means images, and that any of it makes the output more reliable. ⏱️ Chapters 0:00 Four things a failure leaves 0:21 What the AI Tester Dictionary is 0:32 A modality is a kind of information 0:50 Four formats, one representation 1:08 What you can hand over 1:26 One defect, three kinds of evidence 1:50 Both parts pass, the join fails 2:10 Images are one of four 2:35 Three levels to test 3:02 What multimodal buys you 🔑 Key takeaways • A modality is a kind of information — text, image, audio, video. A multimodal model takes several of them at once, rather than needing everything converted to text first • For a tester that means handing over the screenshot and the recording instead of a written description of what you saw in them • It introduces a failure that lives between the inputs: the model reads the text correctly, reads the image correctly, and still gets the relationship wrong • So test all three levels: each input alone, the join between them, and the answer at the end. Most teams only ever check the third 📚 In this series #001 — Artificial Intelligence #002 — Machine Learning #003 — Deep Learning #004 — Generative AI #005 — Large Language Models #006 — Transformer #007 — Diffusion Model #008 — Foundation Model #009 — Token #010 — Context Window #011 — Prompt #012 — System Prompt #013 — Prompt Engineering #014 — Temperature #015 — Top K and Top P #016 — Hallucination #017 — Embeddings #018 — Vector Database #019 — Semantic Search #020 — Retrieval-Augmented Generation #021 — Chunking #022 — Reranking #023 — Knowledge Graph #024 — Multimodal AI ← you are here #025 — Function Calling #001 to #016 are Level 1, AI Foundations. #017 onward is Level 2, the terms that introduce new things which themselves need testing. ▶️ Previously #023 — Knowledge Graph: storing entities and the named edges between them, and why the edges are the part none of your tools records.    • What Is a Knowledge Graph? Explained For S...   ▶️ Next in the series #025 — Function Calling: how a model stops writing text and starts doing something. 🔔 Subscribe for practical AI and software testing content. #SoftwareTesting #AIForTesting #AITesterDictionary #MultimodalAI #VisionLanguageModel #QAEngineer #SDET #AITesting #VisualTesting #TestAutomation #QualityEngineering #SoftwareQuality FOLLOW ME 📸 Web: https://softwaretestingtrends.com/ Discord:   / discord   LinkedIn:   / ambysan   Twitter:   / ambysan   Facebook:   / softwaretestingtrend   Medium:   / ambysan   BLOGGING 📝 Software Testing Trends: https://softwaretestingtrends.com/ TechJournal: https://ambreenkhan.dev/ ABOUT ME 👩‍💻 https://ambreenkhan.netlify.app/